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ABSTRACT 


The Navy small arms training and qualification instruction focused on dry fire and 
simulators should be utilized when available. However, naval personnel often do not have 
access to dry fire training opportunities and therefore may be at risk for losing perishable 
marksmanship skills. The purpose of the study was to determine if the use of a simulator 
is at least as effective in marksmanship training as traditional dry fire techniques. 

A between-groups study with a random selection of volunteers (blocked by 
previous marksmanship experience) was conducted to determine if participants who 
completed simulation-based training showed greater improvement and retention of 
marksmanship skills than participants completing dry fire training. The main measures 
were the qualification scores, average shot group size, shot group size and mean point of 
impact to center zeroing point length. The experiment utilized a simulated M9 Berretta 
for the qualification. 

Results demonstrate that simulation training improved performance on the seven- 
yard line to a greater extent than current naval training. Exploratory analyses suggest that 
simulation training may be most beneficial for less-experienced shooters, and that a 
minimum of two weeks’ time is needed to detect group differences in the retention of 
skills. 
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I. INTRODUCTION 


A. STUDY PURPOSE 

The purpose of this thesis was to determine whether simulation-based 
marksmanship training with the Indoor Simulated Marksmanship Trainer (ISMT) 
improves marksmanship performance to a greater extent than current naval 
marksmanship training. This research was intended to provide information on the 
performance of a shooter to meet a qualification standard and on the retention of skills 
over time if basic marksmanship training was conducted with the use of a simulator. Any 
benefits on different methods of training and the quality of the feedback to a trainee can 
preserve a perishable skill like marksmanship. Naval commands could use the 
information to effectively maintain gun qualifications for inport duty section watch bills 
and constant anti-terrorism readiness. 

The motivation for this thesis was due to the author's personal experiences of the 
lack of hands-on weapons training prior to a weapon qualification. The majority of the 
naval watch standers only handle a gun when they check in and out of the armory. Sailors 
may know what needs to happen during force protection exercises but may not have the 
marksmanship skills required to effectively employ their weapon for deadly force. The 
vessel, board, search and seizure (VBSS) team endured an extensive training program but 
the majority of ship’s crew does not receive in-depth marksmanship training or 
experience with live fire. In many cases, when a sailor failed to qualify on the standard 
navy handgun qualification course of fire (COF) they simply were ordered to the back of 
the line to shoot again. The anti-terrorism watch officer, chief of the guard, and topside 
rovers are a few watch standers that rely heavily on the sailors’ ability to remember pre¬ 
planned responses. There are pre-requisite weapons qualification required to stand those 
watches and the quality of that training is critical. Sailors are required to stay sharp and 
have a high level of knowledge for pre-planned responses to know when to use deadly 
force, but it is equally important to learn, acquire and practice marksmanship skills for 
inherent right of self-defense of any naval unit. 
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This study attempted to identify any benefits in training techniques and 
procedures for improving marksmanship performance via ISMT. The Navy uses 
marksmanship simulators at shore facilities for sustainment qualification, at a gun range 
or conducts a gun shoot on the flight deck while underway in order to ensure gun 
qualification cards are current for all watch standers. Surface Warfare Officer School 
(SWOS) in Newport, Rhode Island exposes department head students to a marksmanship 
simulator that goes over several security scenarios. However, it is uncommon for naval 
vessels to have a dedicated compartment for a simulator to determine improvement in 
watch stander marksmanship performance. Ships compile a list of shooters based on 
expiration of gun qualifications and periodically make the flight deck ready for live-fire 
training exercises while underway. Any practice prior to a gun qualification is 
nonexistent due to ammunition onboard, facilities available and/or time constrains. 
Familiarity with weapon capabilities is based on duty section training and it is uncommon 
to include dry fire training at the same time. Figure 1 is a picture the author took of his 
commanding officer during a pistol qualification. 



Figure 1. Commander J. L. Killman, Commanding Officer of USS Elrod (2012- 
2013) and GM1 Clay as the range safety officer during a pistol 
qualification on April 10, 2013. 
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A qualification conducted while underway adds extra challenges to a sailor 
because the sea state may create significant pitch and roll that will affect the shooters 
stability as the flight deck shifts. The targets are weighted down and tied off however the 
weather could cause the targets to sway if the wind is strong enough. The shooters must 
adapt and overcome these factors to qualify. By using simulation-based training, sailors 
can re-enforce their marksmanship skill with repetitive weapon handling drills and 
receive feedback on accuracy. This technology can be used to add training value to a 
watch stander onboard a ship regardless of geographic location or time in the current 
training cycle. 

Commander Naval Surface Forces Atlantic (COMNAVSURFLANT) released a 
naval message, with date time group of R 271735ZFEB14, for a small arms live fire and 
simulator facility information request. Paragraph 1 of this message reads 
(COMNAVSURFLANT, 2014). 

Feedback from the fleet indicates a need to analyze the capabilities of 
navy ranges and small arms weapons simulators (SAWS). In order to 
determine if efficiencies in scheduling, throughput, operational hours, 
physical expansion and new construction of small arms ranges are 
required to meet operational requirements, (p. 1) 

With the increase in operational tempo, the U.S. Navy has had little opportunity 
for sailors to receive training and practice on a live range prior to qualifications. As a 
result, there has been limited training afforded to sailors who must receive a passing 
qualification score in order to stand a force protection watch that required a gun. 
COMNAVSURFLANT further states (COMNAVSURFLANT, 2014): 

...Although Commanders have done an excellent job qualifying 
personnel, this practice has resulted in insufficient data to make informed 
operational, budgetary, and resource allocation decisions with regard to 
range capabilities, limitations, scheduling, maintenance, and the 
optimization of training, (p. 1) 

The ISMT is a portable stand-alone marksmanship trainer that currently is used to 
help train U.S. Navy and Marines. The ISMT is designed to train sailors and marines in a 
classroom setting and uses various weapons such as the M9 and M16A2 as its training 
weapons. It has the capability to train up to four individuals and provides immediate 
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feedback to the instructor and trainee on weapon trigger pull, cant position, barrel 
movement, rifle butt pressure, tracing of the muzzle on a weapon prior to and after a shot, 
and impact grouping. This feedback enables the participants to adjust their weapon 
accordingly to improve accuracy. More importantly, this feedback allows the instructor to 
effectively evaluate the individual’s technique because the muzzle movement is recorded 
and displayed for review under a trace profile feature. The human eye may not be able to 
see what happens to the weapons when the participant pulls the trigger. This tracking 
technology allows instructors to take advantage of improving the feedback to the shooter 
for better performance. 

The focus of this thesis is to provide data regarding the improvement of training 
via ISMT. In order to have an effective training program, marksmanship training needs to 
be divided into three phases: learning what shooting a weapon entails (breathing, etc.); 
understanding the basic knowledge of weapons handling, techniques, and fundamentals; 
and execution of skills acquired on a range (Chung, Delacruz, Vries, Bewley, & Baker, 
2006). Gunners mates (GMs) and master at arms (MAs) are generally proficient on the 
first phase because there are several power point tutorials and training instruction that 
cover what shooting entails. Training weapons are used for the second phase, basic 
knowledge of weapons handling, techniques and fundamentals; this training typically 
happens a few months prior to an antiterrorism or force protection certification. 

The third phase, execution of skills, is critical, yet is the most lacking in the 
program in the experience of the author. In the author’s experience, sailors are expected 
to execute skills acquired on a range if a real-world threat presented itself. However, 
sailors only handle weapons during watch turnover on duty days that occur once or twice 
a week depending on the number of duty sections. Sailors do not shoot again until their 
qualification is about to expire in order to maintain the number of duty sections. As 
previously stated, sailors that fail to meet the marksman category are sent to the back of 
the line for a second attempt to qualify. If sailors qualify as a marksman or sharpshooter, 
they are expected to carry out their duties and are not allowed to shoot again. There is no 
expectation that a sailor needs to perform better or to continue to practice any marksman 
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skills and techniques. It is this third phase in which simulation training may be 
particularly useful. 

B. LITERATURE REVIEW ON SIMULATED MARKSMANSHIP 
TRAINING 

Below, literature regarding the use of simulation in marksmanship training is 
reviewed. The focus of the literature review is the few studies that have compared 
simulation training in marksmanship to live training. Emphasis is placed on studies that 
specifically examined the ISMT. Baker et al. (2004) conducted research for rifle 
marksmanship performance focusing on predicting the shooting performance of 
participants utilizing advance distributed learning environments (i.e., sensing apparatus). 
Her research showed evidence to suggest a base knowledge component to shooting 
performance and differences in knowledge of marksmanship. Using the advanced 
distributed learning environment, she was able to predict the participant’s marksmanship 
scores regardless of the participant’s prior performance scores (Chung et al., 2006). 

Scribner, Wiley and Harper (2007) conducted a training transfer study using 
dismounted infantry survivability and lethality test-bed (DISALT) for simulation-based 
training to determine differences between the dependent performance and subjective 
measures for a live versus simulated pop-up target shooting with a rifle. They used 12 
Army soldiers with a military occupational specialty of indirect fire dismounted 
infantryman. In their experiment, each participant was exposed to ten trials in a 
simulation facility and 10 trials on a live fire range (Scribner et al., 2007). Half of the 12 
Amy soldiers completed trials in the simulator and then on the live fire range. The other 
half completed their trials in the opposite order. The results showed no significant 
difference in the main marksmanship score or hit percentages between the two different 
types of exercises. In other words, simulation performance was equivalent to live fire 
performance on main measures of performance. However, differences did emerge on 
more subtle measures of marksmanship performance: reaction time and radial aiming 
error. Reaction time was longer and the radial error was lower in the simulated 
environment. When they were engaged in live fire, they experience more of the elements 

when compared to people in a simulator and thus react more quickly. Human attributes, 
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such as rifle movement with a live weapon and rounds, may change the psychological 
dynamic of the scenario (Scribner et al., 2007). Because of these results, this thesis 
focused on the use of a simulated 9mm Beretta to prevent this human attribute from 
affecting the study of standard Navy marksmanship training and simulation-based 
marksmanship training. 

Two studies that specifically focused on the use of ISMT for training were written 
by Major William Yates (2004) and by Lieutenants Timothy Jensen and John Woodson 
(2012). Both studies examine training transfer, but Yates used a M16A rifle while Jensen 
and Woodson used a 9mm Beretta. Yates’s study contained two experiments the first 
experiment evaluated and quantified the effectiveness of the ISMT in training novice 
shooters in the technical operation of the M16A service rifle. The second experiment 
assessed whether task performance in the ISMT is of predictive value with respect to the 
actual task performance using live ammunition on the Marine Corps’ standard known 
distance course of fire (Yates, 2004). 

The participants for the first experiment were an all-male platoon of Marine Corp 
recruits at Edson Range on Camp Pendleton Marine Corps Base, California. The control 
group had no knowledge of the ISMT and conducted training with a technique referred to 
as “snapping-in.” Snapping-in ordered the participants to assume the shooting position, 
aim an unloaded rifle with an empty magazine and “dry-fire.” Dry-firing is the actuation 
of the trigger without any ammunition being loaded in the weapon (Yates, 2004). The test 
group had approximately nine hours of training with ISMT. The control group and test 
group both utilized the same instructor and had 55 participants in each group. 

The second experiment provided insight on the acquisition of marksmen skill with 

repetitive practice on the ISMT without the presence of a skilled coach. The 28 

participants included civilians with no affiliation to the Department of Defense (DOD) 

and military officers assigned as students and civilian employees at the Naval 

Postgraduate School (Yates, 2004). Thirteen subjects completed at least seven trials. The 

other 15 participants completed four or fewer trials. Every participant was characterized 

as either trained or untrained based on any formal marksmanship instruction. This 

experiment measured the consistency of their aim based on the average diameter of their 
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shot groupings. They were expected to fire two strings of three rounds each and the third 
string fired four rounds. The furthest outlying round of the third string of fire was 
discarded. These participants lacked the advice of a skilled coach but were briefed on 
how to use the rifle, fundamentals of good shooting position, weapon support, sight 
picture and alignment, trigger squeeze and breather control (Yates, 2004). 

The results from the first experiment show equivalent performance in 
marksmanship scores of recruits trained on ISMT versus recruits in a control group 
without ISMT training. It is important to note that the test group experienced foul 
weather condition on the last three days that led up to the qualification day, while the 
control group experienced optimal weather condition all five days leading up to 
qualification day. Yates’s study was able to prove that subjects with ISMT training does 
not result in negative training transfer (2004). Additionally, there was a positive 
correlation between simulated qualification COF scores and live fire qualification COF 
scores (Yates, 2004). The second experiment found no evidence to support that untrained 
participants produced steady improvement using ISMT without a coach (Yates, 2004). 
However, a trained shooter retained proficiency and showed small increments of steady 
improvement without a coach (Yates, 2004). 

The results above are somewhat ambiguous, but they indicate that simulator 
training can lead to equivalent performance as live fire training for certain tasks, and 
therefore, may be a good alternative when live fire training is unavailable. People may 
not be ready to replace live fire training with simulated marksmanship training but these 
advances in marksmanship simulators are vital to improving the performance of a shooter 
through part task training in a virtual environment. Of note, the above experiments used a 
rifle as the weapon of choice. A comparison of simulator and live fire training on other 
weapons, such as pistols is needed. 

Jenson and Woodson (2012) informally interviewed approximately 50 Navy 

officers that served in billets located near the 41 ISMT systems across the fleet and only 

one person knew ISMT existed. This finding pinpoints the current problem in Navy 

marksmanship training. By instruction, sailors are expected to use simulators to aid in 

marksmanship training, yet the Navy commands are not utilizing available assets or 
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ISMT training facilities are not supporting the fleet. The purpose of their study explored 
whether simulation-based marksmanship training with ISMT transfers to live fire and 
provided insight into how long beneficial effects of simulation training lasted (Jenson & 
Woodson, 2012). Their study used 34 participants of active duty military or faculty at 
Naval Postgraduate School in a two (control versus simulated training) by two (two or 
four week gap between training and live fire qualification). Thus, there were four groups: 
two control groups trained with standard Navy marksmanship training and the two test 
groups trained with simulation training on ISMT. One control and test group received no 
additional training after two weeks, and the other control and test group received no 
additional training after four weeks. All four groups conducted a final live fire Navy 
handgun qualification course that determined retention of marksmanship knowledge. 
Importantly, participants were blocked into the groups by self-reported previous 
marksmanship experience. 

The training programs provided were baseline training, control group 
marksmanship fundamental training and simulation group marksmanship fundamental 
training. All participants received baseline training and covered safety, weapon 
conditions and commands, training environment familiarity, and approved firing position. 
The control groups marksmanship fundamentals covered dry fire procedures, aiming, 
sight alignment, proper grip and control, trigger pull, breather control and practiced dry 
fire training. The simulation group marksmanship training covered the same topic but 
excluded the dry fire procedures because the simulated 9mm Beretta was connected to 
ISMT and a carbon dioxide (C0 2 )bottle in order to provide video and audio feedback and 
simulated recoil. 

The results comparing performance at baseline qualification to live fire 
qualification found that simulation groups showed significantly less degradation in mean 
point of impact (MPI) performance than the control group. The MPI provides a more 
accurate measure of marksmanship performance than score and therefore may be more 
indicative of how well a watch stander would perform in an actual threat situation 
(Jenson & Woodson, 2012). It is important to note that the simulation group achieved 
better MPI performance than the control group at the seven- and 15-yard distance during 
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post training and performed better at the three-yard distance for live fire. The results on 
retention show no significant difference in either two or four week gaps. All groups 
showed a pattern of performance degradation since their post-training but also showed 
improvement from their baseline qualification to the live fire qualification. 

The Navy could benefit from a study that compares current naval marksmanship 
training and simulation-based marksmanship training. The experiment presented in this 
thesis assessed whether ISMT can be used as an effective part-task trainer to improve 
marksmanship performance for the 9mm Berretta on the standard navy handgun 
qualification COF compared to current naval marksmanship training. Like the Jenson and 
Woodson (2012) study, participants in this thesis were categorized based on their 
qualification score in accordance with standard Navy Handgun Qualification COF (Chief 
of Naval Operations, 2009). The categories are as follows: novices, marksman, 
sharpshooter, or expert prior to training on the ISMT (Clark, 2010; Chief of Naval 
Operations, 2009). This thesis also assessed how well participants retained 
marksmanship skills over a one-week period. Pistol marksmanship training was the 
focus of this experiment because the majority of the naval security watch standers do not 
carry rifles. The next section explains the details of the training programs created for this 
experiment. 

C. MARKSMANSHIP TRAINING BACKGROUND 
1. Training Requirement 

The current naval training requirements are specifically addressed in the small 
arms training and qualificaiton instruction within enclosure three, paragraph five (Chief 
of Naval Operations, 2009). 

Before each qualification fire and sustainment training session, all 
shooters shall receive instruction on marksmanship, safety and weapon 
familiarization... Shooters shall also practice drawing from the holster, 
and must demonstrate the ability to safely handle and present the weapon 
to the instructor before live firing. Shooters must also demonstrate 
knowledge of the four general safety rules, weapons commands, and 
weapon condition codes before live firing... (p. 2) 
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All of the required training can be found in the separate enclosures of the small 
arms training and qualificaiton instruction (Chief of Naval Operations, 2009). Enclosure 
one covers the range regulations, general safety rules, general range safety rules, range 
operations, weapons commands, pistol safety rules, and training time out. Enclosure two 
covers the procedure to properly conduct dry fire training and enclosure three explains 
the qualification criteria for pistols and the navy handgun qualification course of fire 
(Chief of Naval Operations, 2009). Enclosure three was used to ensure that the 
qualification course of fire (COF) scenario created in ISMT was within the current naval 
standards. This thesis also used the Force Protection Weapons Handling Standard 
Procedures and Guidelines (U.S Fleet Forces Command, 2003), Pistol Marksmanship 
(Headquarters, United States Marine Corps, 2003), the Combat Training with Pistold, M9 
and Mil (Headquarters, Department of the Army, 2003). These reference were used to 
enhance the training provided by the small arms training and qualificaiton instruction 
(Chief of Naval Operations, 2009). 

D. RESEARCH QUESTIONS AND HYPOTHESIS 

1. Question One 

Can the ISMT be used as an effective part-task trainer to improve marksmanship 
performance for the 9mm Berretta on the standard Navy handgun qualification COF? 

Null hypothesis (HO): Participants who receive simulation-based marksmanship 
training in ISMT will have the same amount of improvement, from the baseline 
qualification to the training qualification, in marksmanship performance and scores on 
the standard navy handgun qualification COF than the participants who received current 
naval marksmanship training. The control group and the experimental group will have 
the same level of improvement regardless of the marksmanship training provided. 

Alternate hypothesis (HA): Participants who receive simulation-based 
marksmanship training in ISMT will have a greater improvement, from the baseline 
qualification to the training qualification, in marksmanship performance and scores on 
the standard navy handgun qualification COF than the participants who received current 
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naval marksmanship training. The experimental group will improve more than the control 
group as a result of the marksmanship training provided. 

2. Question Two 

If a participant in the experimental (simulation) group does not pass the baseline 
standard navy handgun qualification COF, will they have a better chance of meeting the 
marksmanship standard after receiving simulation-based training in ISMT than a 
participant in the control group? 

HO: Among those who do not meet the qualification standard for the standard 
navy handgun qualification COF at baseline, there will be an equal number of 
participants in each group who do not meet the qualification standard after their 
simulated marksmanship training. Those in control group and the experimental group 
that fail the baseline COF will have the same level of improvement regardless of the 
marksmanship training provided. 

HA: Among those who do not meet the qualification standard for the standard 
navy handgun qualification COF at baseline, a greater number of participants in the 
experimental group will reach the qualification standard after simulation-based 
marksmanship training than in the control group. Those in the experimental group that 
fail the baseline COF will improvement more than those control group that fail the 
baseline COF as a result of the marksmanship training provided. 

3. Question Three 

Will participants in the experimental (simulation) group have a better chance of 
retaining what they learned after one week of no instruction, than participants in the 
control group? 

HO: Participants in each group will be equally likely to maintain their 
marksmanship performance and scores on the retention standard navy handgun 
qualification COF one week after the training qualification. Those in the control group 
and the experimental group will retain the same level of marksmanship skill after a week 
of no training. 
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HA: Participants in the experimental group will be more likely than those in the 
control group to maintain their marksmanship performance and scores on the retention 
standard navy handgun qualification COF one week after the training qualification. 
Those in the experimental group will retain a higher level of marksmanship skill than the 
control group after a week of no training. 

4. Overview of Thesis 

A demographic survey was administered to all participants prior to the beginning 
of the experiment (see Appendix H). A baseline qualification assessed their current level 
of marksmanship performance. The participants’ baseline scores along with their reported 
previous experience on the demographic survey were used to block the participants into 
the control and experimental groups. The performance measurements and score on the 
baseline qualification were recorded before the participants received any simulated 
marksmanship training. The second handgun qualifications were administered after the 
control and experimental groups receive their designated simulated marksmanship 
training to establish any main effects from their baseline. Participants did not receive 
additional training for a week and the instructor administered a final handgun 
qualification to establish retention of marksmanship skills. 

E. THESIS ORGANIZATION 

The introduction chapter provides the rationale behind this thesis. The literature 
review gives focused overview of studies in the field of marksmanship and how previous 
work lead up to this thesis. The background information on marksmanship training is an 
explanation of the current naval standards, and naval training requirements .The research 
questions and corresponding hypotheses are listed, as well as an overview of the 
experiment. 

The methods chapter explains the research design, demographic characteristics of 

the participants, equipment and scenarios, main measures, and procedures. The research 

design section provides a depiction of independent and dependent variables followed by a 

list of controls used to conduct the experiment. The next section goes into the process of 

how participants were recruited and a description of how they were split up into either the 
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control group or experimental group. The methods chapter closes with an explanation of 
the procedures to conduct the experiment and provide enough detail for future thesis 
students to repeat the same work. 

The results chapter provides the results of the experiment starting with a broad 
look at the descriptive statistics and then describing the results for each hypothesis. The 
discussion chapter leads into an interpretation of the findings based on the significant and 
non-significant results. 

The last chapter is the conclusions and recommendations. This chapter focuses on 
any shortcomings and how to improve future work. This chapter also identifies all ISMT 
technical difficulties with the software while creating the naval standard handgun 
qualifications scenarios and the malfunctions with seven of the M9 Berettas. 
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II. METHOD 


A. STUDY OBJECTIVES 

The objective of this thesis was to determine any changes on marksmanship 
performance based on the type of training a participant received. The ISMT was used as a 
part task trainer to help participants in the experimental group learn how to fire a pistol 
one step at a time. Dry fire training was conducted with the control group for the same 
purpose of teaching marksmanship. The experiment determined if simulation-based 
training conducted in ISMT was able to increase marksmanship performance to a greater 
extent than a training program that meets current naval marksmanship requirements. 

1. Research Design 

The experimental design for this thesis utilized between-groups comparisons in 
which participants were blocked by previous marksmanship experience. The independent 
variables were the different training programs: the control group and experimental group. 
The control group received current naval marksmanship training that went over 
fundamentals of marksmanship followed by untimed dry fire and a practice qualification 
that implemented time limits. The experimental group received simulation-based 
marksmanship training that consisted of fundamentals of marksmanship followed by 
untimed simulation practice and a practice qualification that implemented time limits. 

2. Training Methods Used 

A brief overview of the procedure was administered to the participants before 
they received what was referred to as baseline training in this thesis. The participant then 
received their baseline qualification. Enclosure one of the small arms training and 
qualificaiton instruction (Chief of Naval Operations, 2009) was used to ensure that the 
safety training requirements were met. The purpose of this training was to establish a 
basic level of knowledge for participants that have never fired a weapon. All the 
participants were from the Naval Postgraduate School (NPS); however the level of 
marksmanship skill at the beginning of the experiment will drastically vary depending on 
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if a participant is a civilian (either a NPS student or facility and staff) or active duty 
military (assigned to NPS). The participants that were active duty could still have a lot of 
variability in their skill level because of the different services. 

With all the many possibilities, the baseline training program ensured that at a 
minimum, everyone started with a basic understanding of the requirements for this 
experiment. The training covered how to use the M9 Berretta, and explained the specific 
weapon commands used in the experiment. Refer to Table 1 for the content of the entire 
baseline training program. 


Baseline Training 

General Range Safety Rules 
General Safety Rules 
Weapon Conditions 
Weapon Commands 
Pistol Safety Rules 
Ready Line 
Firing Line 
Range Operations 
Training Time Out: 

Loadimgthe Pistol 
Makingthe Pistol Ready 
Fire 

Cease Fire 

Unload, Showingfhe Pistol Clear 
Dry Reload 
Isosceles only 
Isosceles Standing Position 
Isosceles High Kneeling Position 

Table 1. Baseline Training Program 

It is important to note that no marksmanship fundamental training was provided because 
the purpose of the baseline qualification was to capture the level of marksmanship skill 
that each participant had at the beginning of the experiment. 

The second training programs that the participants received were used to 
determine if there were any changes from the baseline qualification. The second chapter 
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of both the Force Protection Weapons Handling Standard Procedures and Guidelines 
(U.S Fleet Forces Command, 2003), and the Pistol Marksmanship (Headquarters, United 
States Marine Corps, 2003) were also used to establish the baseline training program. It is 
also essential to note that the last part of the baseline training program specifically 
addressed the isosceles firing positions. Refer to Figure 2 for a picture of the isosceles 
standing position and refer to Figure 3 for a picture of the isosceles high kneeling 
position. 



Figure 2. Isosceles Standing Position from Pistol Marksmanship (from Headquarters, 

United States Marine Corps, 2003) 
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Figure 3. Isosceles High Kneeling Position from Pistol Marksmanship (from 
Headquarters, United States Marine Corps, 2003) 


According to the small arms training and qualificaiton instruction (Chief of Naval 
Operations, 2009) the isosceles and Weaver firing positions were acceptable. The Weaver 
stance was not allowed in this thesis because of the difficulty in maintaining controls for 
the experiment. The instructors would have had a difficult time trying to ensure that 
every participant assumed the exact same Weaver standing position every single time. 
The Weaver high kneeling position would have provided so much support that it would 
have been more difficult to determine if the participant was actually utilizing the proper 
breathing technique, sight picture and sight alignment. The isosceles high kneel position 
would have provided enough support for the participant to maintain their balance. It 
would have also been just enough instability to make it easier to determine if the 
participant used proper marksmanship fundamentals. Refer to Figure 4 for a picture of the 
Weaver standing position and Figure 5 for a picture of the Weaver high kneeling 
position. 


18 




Figure 4. Weaver Standing Position from Pistol Marksmanship (from Headquarters, 

United States Marine Corps, 2003) 
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Figure 5. Weaver High Kneeling Position from Pistol Marksmanship (from 
Headquarters, United States Marine Corps, 2003) 

The control group training program was created by using enclosure two of the 
small arms training and qualificaiton instruction (Chief of Naval Operations, 2009), 
Chapter 2 of the Force Protection Weapons Handling Standard Procedures and 
Guidelines (U.S Fleet Forces Command, 2003), and Chapter 3 of the Pistol 
Marksmanship (Headquarters, United States Marine Corps, 2003). The participants were 
allowed to handle the M9 Berretta and follow along with the recorded lessons. It was 
important that the participants were afforded the opportunity to practice without any time 
constraints. During the lecture, the participants focused on slow, smooth and methodical 
motions when drawing and dry firing the pistol. The participants in the control group 
received 18 minutes of recorded lecture. Refer to Table 2 for a complete list of the items 
covered in the lecture. 
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Control Group Training 

Dry Fire Safety Training 
Drawing Process 
Grab 
Draw 
Smack 
LookSqueez 
Holstering 

Introduction to Fundamentals 
Aiming 

Sight Alignment 
Establishing Sight Alignment 
Grip with respect to (WRT) Aiming 
Controlled MuscularTension 
Sight Picture 

Relationship Betweent the Eye and the Sights 
Trigger Control 

Sight Alignment and Trigger Control 
Grip WRT Trigger Control 
Trigger Finger Placement 
Uninterrupted Trigger Control 
Interrupted Trigger Control 
Breath Control 
Begin Dry Fire Training 
Single Action 

Five to Ten Times Repitition 
Start Adding Time Limits 
Draw Sweep Safety Dry Fire 

Table 2. Control Group, Marksmanship Fundamental Training Program 

A practice qualification was administered to the control group after completing 
the marksmanship fundamentals lecture. The M9 Berretta was disconnected from the 
ISMT which prevented the use of carbon dioxide (CO 2 ) and only dry fire training 
techniques was used. The purpose of this practice was to implement time constrains and 
to build the participants’ expectations towards a navy handgun qualification. There was a 
pause between trials at the three, seven and 15-yard line. This pause was provided to the 
instructor in case any training was required for the participant to maximize training 
needs. Upon completion of the practice qualification, no further training was administer 

for the duration of their involvement with the experiment. A second qualification was 
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administered to the participants in the control group to determine any main effects as a 
result of the current naval marksmanship training program. After a week of no further 
training the participants came back on the same day of the week and time of day for a 
third qualification. This qualification was done to determine if any of the participants 
retained any marksmanship skill. 

3. Use of ISMT 

The experimental group training program was created in a similar manner, but 
without using enclosure 2 of the small arms training and qualificaiton instruction (Chief 
of Naval Operations, 2009). Instead of conducting dry fire during the marksmanship 
fundamentals training lecture, they received simulation-based training that provided 
instant feedback from the ISMT. The current simulator training requirements for the navy 
are specifically addressed in the small arms training and qualificaiton instruction within 
Enclosure three, paragraph 5 (Chief of Naval Operations, 2009). 

Where available, simulator training shall include marksmanship training 
drills to assist instructors in identifying and correcting personnel shooter 
fundamental errors. Courses of fire should be programmed into the 
simulator to provide personnel the opportunity to simulate firing the COF 
prior to live fire operations, (p. 3) 

The participants received 16 minutes of recorded lecture. The control group 
lecture was 2 minutes longer because it took longer to read the requirements for dry fire 
versus simulation training requirements. The difference in time did not cause any drastic 
differences between the two groups. The experimental group replaced the time lost in the 
lecture with time to review trace profiles from their baseline qualification during the 
pauses in between stages of the practice qualification. Refer to Table 3 for a complete list 
of the items covered in the experimental group, marksmanship fundamentals lecture. 


22 



Simulation Group Training 
Simulation Safety Training 
Drawing Process 
Grab 
Draw 
Smack 
LookSqueez 
Holstering 

Introduction to Fundamentals 
Aiming 

Sight Alignment 
Establishing Sight Alignment 
Grip with respect to (WRT) Aiming 
Controlled MuscularTension 
Sight Picture 

Relationship Betweent the Eye and the Sights 
Trigger Control 

Sight Alignment and Trigger Control 
Grip WRT Trigger Control 
Trigger Finger Placement 
Uninterrupted Trigger Control 
Interrupted Trigger Control 
Breath Control 
Begin Simulation Training 

Table 3. Experimental Group, Marksmanship Fundamental Training Program 

The participants were allowed to handle the M9 Berretta and follow along with 

the recorded lessons. During the lecture, the participants focused on slow, smooth and 

methodical motions when drawing and dry firing the pistol. The simulation group 

received immediate feedback from the ISMT because the weapon displayed where the 

shots were fired on the target. They experienced the simulated recoil from the CO 2 tanks 

and the ISMT simulated the sound of a pistol being fire as audio feed back. The training 

scenario created for this phase of the training is called unlimited practice and its purpose 

is to provide the participants with information identifying which shot was the last one 

fired and displayed a value associated with that last shot. The scenario permanently 

displayed three targets to the participant to represent the actual size of the three, seven 

and 15 yard line targets. The control group saw the same scenario for the same phase of 

their dry fire training except the M9 Berretta was disconnected from the ISMT. 
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A practice qualification was administered to the simulation group upon 
completion of the marksmanship fundamentals training lecture. The practice qualification 
paused in between the three, seven and fifteen yard line stages of fire. Unlike the control 
group, the simulation group had an opportunity to review 48 trace profile pictures before 
they shot the practice qualification. These trace profile pictures were recorded after their 
baseline qualification. The participant viewed all twelve shots fired at the three-yard line 
and tried, with the instructor, to gather any information addressing marksmanship 
fundamental errors. Then the participants shoot the three-yard line stage of the practice 
qualification. The ISMT scenario was paused to give the instructor time to present the 
next 12 trace profiles pictures of shots fired at the seven yard line from the baseline 
qualification. After the participant learned from the instructor’s comments about the trace 
profile pictures they shot the seven-yard line stage of the practice qualification. The same 
process applied for the pause between the seven and 15-yard line. 

Upon completion of the practice qualification, the participant had an opportunity 
to review the three-, seven- and 15-yard line shot group data for the practice qualification 
they just fired. This data provided feedback of their performance and the instructor 
identified any errors that can be addressed before the second qualification was 
administered to the participant. Once the second qualification began the participant no 
longer received any assistance from the instructor and had to rely of what they learned. 
After a week of no further training the participants came back on the same day and time 
for a third qualification. This qualification session was administered to determine 
participants’ level of marksmanship skill retention. 

All participants completed three qualifications. The first provided baseline 
marksmanship skills. The second was used to determine the effect of the training. A third 
qualification was administered to both groups exactly one week after the completion of 
training to determine if the participants retained their marksmanship skills. The 
dependent variables were the various marksmanship performance data that ISMT 
provided at the end of each qualification. Marksmanship performance was measured by 
diameter size of average shot group, diameter size of shot group for the three, seven and 
15 yard line, length of mean point of impact to center zeroing point (MPI to CZP) for the 
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three-, seven- and 15-yard line, and scores on the standard navy handgun qualification 
COF. 


Research Design 


Controls 

55 Minutes Max 

per session 

Recorded 

Course of Fire 

Instructions 

Isosceles 

Standing 

Position 

Isosceles 

High 

Kneeling 

Position 

i Week 

Between 2nd 

and 3rd dual 

Baseline and 

Fundamentals 

Training 

Security 

Belt with 

Holster 

Firing Line 

M9 Bereft a 

Independent 

Variable 

Current Naval 
Marksmanship 
Training 

Simulation 

Based 

Marksmanship 

Training 








Dependent 

Variable 

Average Shot 
■Group Size 

Qualification 

Score 

Skill Level 

Rating 

3 Yard Line 

Shot Group 

Size 

7 Yard Line 

Shot Group 

Size 

15 Yard Line 

Shot Group 

Size 

3 Yard Line 

MPI to CZP 

7 Yard Line 

MPI to CZP 

15 Yard Line 

MPI to CZP 


Table 4. Research Framework 


4. Standardization of Methods and Reduction of Confounds 

There were several controls in place to ensure standardization among the 
participants and to prevent any foreseeable confounds. Refer to Table 4 for a synopsis of 
the main variables of interest, and the factors that were controlled in the experiment. The 
majority of the participants were students and the maximum allotted time per session was 
55 minutes. This time duration allowed students to participate in the study without 
interfering with any class schedule. To ensure that one group did not get an unfair 
advantage, close attention was paid to the amount of training time. 

Additionally, the author conducted a class project (i.e., a pilot study) to learn how 
to use ISMT and to refine the experiment for this thesis. In the class project, the 
instructors read the instruction for the qualification COF and conducted the 
marksmanship training in between the baseline (first) and training (second) qualification. 
A retention (third) qualification was not part of the class project. The author determined 
that recording all the instructions for the COF and all the marksmanship stimulus training 
ensured that the participants were not affected by an inconsistent instructor. The ability to 
repeat the same playlist ensured that one group did not have an advantage over the other 
because of different teaching styles. Prerecorded media files also assisted the instructor 
with managing fatigue and lose of vigilance. 
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The decision to make every participant utilize the isosceles firing positions was an 
attempt to avoid any confounds between different stances. It was difficult to say that a 
person does not have an advantage over another simply because one person decided to 
use a Weaver firing position instead of the isosceles firing position. The isosceles 
standing position was easier and ensured that every participant was consistent. The 
shoulders had to be square to the target at all times and the feet had to be approximately 
shoulder width apart every time. It was very difficult to ensure that every participant had 
the same Weaver standing position, because there was no way to determine that each 
person had their body turned at the same angle every time. To reduce the amount of 
variability in this experiment the Weaver firing positions (standing and kneeling) were 
not allowed. 

The isosceles high kneeling position was an attempt to ensure that every 
participant will assume the same position every time. A kneeling position was more 
important because the author wanted to make sure the participants were implementing 
their marksmanship skills that they acquired from the marksmanship stimulus training. 
The pistol is constantly moving and the participants had to focus on their breathing, sight 
picture and sight alignment along with the other fundamentals. In the Weaver high 
kneeling position, the participants could sit on their foot and rest their elbow on their 
knee to try and stabilize their body. 

By forcing every participant to assume the isosceles high kneeling position, the 
author reduced the amount of variability in marksmanship performance that could be 
accounted for by differences in position. This position forced the participants to focus on 
all the elements of marksmanship fundamentals similar to the standing position. If the 
participant failed to maintain proper muscular tension or properly place the finger on the 
trigger then the performance should vary more in the isosceles high kneeling position as 
opposed the being supported from the leading knee and sitting on the back foot with the 
weaver high kneeling position. 

Every participant stayed behind the firing line during training and qualifications. 

They stayed behind the ready line until instructed to move forward to the firing line by 

the instructor. A blue sticker was placed as a guide to help all participants center 
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themselves on the target at all times. Every participant used the same holster, which was 
connected to the security belt, and the pistol rested on the side of the hip next to the 
participants shooting hand. If the security belt did not fit a participant, the instructor 
detached the same holster and attached it to the participant’s personal belt. The author 
assumed that the difference in belt type would not have a significant effect on the study, 
but made sure that every participant used the same style of holster to draw from. 

The participants ensured that before agreeing to the day and time of their second 
qualification that the same day and time was available for the following week. The 
procedure ensured that one group did not have an unfair advantage over the other simply 
because someone waited longer to complete their retention (third) qualification. 
Marksmanship is a perishable skill and the purpose of the retention phase was to 
determine if participants would retain what they had learned. It would not be fair if some 
participants waited two weeks when everyone else had one. 

B. PARTICIPANTS 

1. Establishing Control or Experimental Group 

Participants were recruited by email or in person to volunteer for the experiment. 
The participants received the recruitment email with the demographic survey attached. 
They were instructed to fill out the demographic survey and submit it prior to their first 
qualification. After the participants completed their baseline qualification, the instructor 
reviewed their score and demographic survey and ranked the participant from highest to 
lowest. Participants were then blocked by this rank into the control and experimental 
groups. This process attempted to ensure that each group had similar distributions of 
marksmanship ability. It is important to note that the focus of the experiment was not the 
scores after completion of each qualification, but the difference between the scores on all 
performance measures to show any changes. For example, if a participant started out as 
an expert on the baseline (first) qualification, changes in his/her performance could still 
be detected by the individual performance measures, such as shot groups and MPI to CZP 
lengths. 
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2. Demographic Statistics 

A demographic survey was administered to every participant at the beginning of 
the experiment to try and get an understanding of their level of marksmanship. This 
information along with their baseline score was required in order to assign them to the 
control or experimental group. Refer to Table 5 for descriptive statistics on participants’ 
demographic characteristics. Refer to Appendix H for a copy of the demographic survey. 


Demographic Survey 

Control Group 

Experimental Group 

Age IS-.25 

0.07 

0.13 

Age 26-30 

0.27 

0.27 

Age 31-35 

0.40 

0.33 

Age 36-40 

0.20 

0.27 

Age 41+ 

0.07 

0.00 

Male 

0.87 

0.80 

Right Handed 

0.93 

0.93 

Wear Corrective Lens 

0.13 

0.27 

Civilian 

0.07 

0.27 

Military 

0.93 

0.73 

Navy 

0.47 

0.40 

Army 

0.20 

0.20 

Marine 

0.13 

0.00 

Air Force 

0.07 

0.07 

InterStudent 

0.07 

0.07 

Year of Service ( Mean) 

11.63 

8.47 

Year of Service (SD) 

5.43 

6.51 

Marksmanship 

Experience 


Novice 

0.27 

0.33 

Marksman 

0.33 

0.40 

Sharpshooter 

0.33 

0.07 

Expert 

0.07 

0.20 


Table 5. Descriptive Statistics on Participants’ Demographic Characteristics 

The demographic survey revealed that there were no significant differences 
between the control and experimental group based on age group, gender or dominant 
hand. The experimental group had more participants that wore corrective lens than the 
control group. The author ensured if a participant started the experiment with or without 


28 











their corrective lens that they consistently used/didn’t use their corrective lens through 
the entire experiment. The experimental group also had a few more civilians and self- 
reported novices, but the control group had fewer self-reported experts. It is also 
important to show that the percentages of participants in separate branch of services are 
close, but the control group had more years in service. In summary, the two groups 
differed only in years of service and in the number of self-reported sharpshooters and 
novice. 


3. Participant Agenda 

The purpose of this section is to provide description of the scheduling 
requirements expected from every participant. The schedule also explains the chain of 
events from a day-to-day projection. Refer to Table 6 for the participant schedule. 


Week 1 

Monday 

Tuesday 

Wednesday 

Thursday 

Control Group 
Baseline Qual 

Control Group 
Training Qual 

Simulation Group 
Baseline Qual 

Simulation Group 
Training Qual 

Dayl 

Day2 

Day3 

Day 4 

Week 2 

Monday 

Tuesday 

Wednesday 

Thursday 


Control Group 
Retention Qual 


Simulation Group 
Retention Qual 


Day9 


Day 11 


Table 6. Participant schedule 


(Day 0) A demographic survey was administered to all participants prior to the 
beginning of the experiment so as to be able to block participants into the control and 
experimental groups by previous marksmanship experience. It contained questions that 
asked about the participants’ characteristics, marksmanship skill level and experience. 

(Day 1) The ISMT qualification on the standard navy handgun qualification COF 
was administered to the control group to establish a baseline of their current level of 
marksmanship performance and score. Their baseline qualification score and the 
information on the demographic survey from day 0 were used to block participants into 
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the control and experimental groups. The standard navy handgun qualification COF 
procedure was checked by a subject matter expert (SME) in order to ensure that the 
proper instruction was given. 

(Day 2) The control group received current naval marksmanship training, which 
involves handling a simulated 9mm Berretta, practicing proper techniques for standing, 
breathing, and doing dry fire. ISMT qualification on the standard Navy handgun 
qualification COF was administered to determine the main effects of the training on the 
control group’s marksmanship performance and score. The current naval marksmanship 
training was checked by a SME in order to ensure that the proper training was given. 

(Day 3) The ISMT qualification on the standard navy handgun qualification COF 
was administered to the experimental group to establish a baseline of their current level 
of marksmanship performance and score. Their baseline qualification score and the 
information on the demographic survey from day 0 were used to block participants into 
the control and experimental groups. The standard navy handgun qualification COF 
procedure was checked by a SME in order to ensure that the proper instruction was given. 

(Day 4) The experimental group received simulation-based marksmanship 
training in the ISMT. This training involves handling a simulated 9mm berretta, practice 
proper techniques for standing and breathing. These participants also had the opportunity 
to fire practice shots at simulated targets, analyze their trace profiles and shot groups. The 
immediate feedback provided by ISMT was the only aspect of the training that was 
different from the control group. ISMT qualification on the standard Navy handgun 
qualification COF was administered to determine the main effects of the training on the 
experimental group’s marksmanship performance and score. The simulation-based 
marksmanship training was checked by a SME in order to ensure that the proper training 
was given. 

(Days 5-8) No additional training was provided. 

(Day 9) ISMT qualification on the standard navy handgun qualification COF was 
administered to the control group. This determined how much marksmanship training the 
participant had retained. 
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(Day 10) No additional training was provided. 

(Day 11) ISMT qualification on the standard navy handgun qualification COF 
was administered to the experimental (simulation) group. This determined how much 
marksmanship training the participant had retained. 

During all training and qualification assessments, data regarding participants’ 
performance score, diameter of shot groups, and trace profiles were collected. After the 
participants had left the ISMT lab, a digital camera was used as backup to collect 
performance and score data that was projected on the screen. The participants were not 
video or audio recorded at any time. 

C. EQUIPMENT 

This section describes the ISMT and how it works. The ISMT enclosure holds the 
computer processing unit (CPU), amplifier and a flip up monitor attached to a keyboard. 
The following components are also a part of the ISMT suite; two speakers, projector, 
projector screen, hit camera, simulated M9 Berretta and a CO 2 tank. The CPU 
synchronized all the components in order to run the simulator. The ISMT lab for this 
thesis utilized an additional computer monitor and keyboard so that ISMT can be 
controlled from multiple stations. The CPU sent video data to the projector that projected 
an image on the screen mounted against the wall ahead. The participant fired a round at a 
target on the screen with a M9 pistol that is connected to the CPU and a carbon dioxide 
(CO 2 ) tank. The CO 2 tank released gas that was used to simulate recoil in the pistol. The 
pistol sent an infrared beam out of the barrel and that signal was received by a hit camera 
that is connected to the CPU. The CPU sent an audio signal to the amplifier which is 
connected to two speakers located on the floor, to the right and left of the screen. The 
speakers simulated the sound of a M9 pistol being fired. The CPU calculated the shot 
fired based on the information received from the hit camera and presented the 
marksmanship performance data at the end of the qualification. Refer to Figure 6 for a 
picture of the ISMT major components. The labels were added to the picture to assist 
identifying components. 
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Figure 6. ISMT Major Components (after Schwetje, 2009) 


1. Building Scenarios 

The ISMT used in this experiment was the Marine Corp version. All the 
preinstalled training programs were within the Marine Corp standards. Therefore, the 
author programmed the ISMT to establish the navy handgun qualification standards. The 
following paragraphs will describe each of the scenarios and its purpose. 

a. Zero Scenario 

The zero scenarios for each of the three, seven and 15-yard lines were 
created. In real life, a pistol does not get zeroed like a rifle because the sights do not 
move. But these scenarios ensured the accuracy of the simulated pistol, thus the zero 
scenarios were used. The zero scenarios were also used for quick operational test of the 
M9 Berettas, which assisted with trouble shooting replacement pistols to ensure their 
integrity during the experiment. All zero scenarios have no time limit and have only one 
magazine of three rounds. Refer to Appendix B, C and D for screen captures of every 
setting to recreate the three, seven and 15 yard line zero scenarios. 
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b. Qualification Scenario 


The qualification scenario was the most important because all the data 
collection would come from the results of the qualification at the end of the scenario. 
After a few trials the author determined it was best to have a target to represent the 
distance of three, seven and 15-yard line. The targets would not appear until the recorded 
COF introduction instructions were completed. Then the targets appeared prior to the first 
shot of that particular stage of fire. The targets disappeared after the last cease-fire 
instruction was completed per stage. This allowed the participants to focus on the 
instructions versus the target. Refer to Table 7 for a complete list of all the stages and 
required sequence of fire. 


Navy Handgun Qualification Course 

Yard Line 

Sequence 

Time Limit 

Remarks 

3 Yards 

Draw and fire 2 rounds 

4 seconds 

Strong Hand Supported 

3 Yards 

Draw and fire 2 rounds 

4 seconds 

Strong Hand Supported 

3 Yards 

Draw and fire 2 rounds, reload and fire 2 rounds 

10 seconds 

Strong Hand Supported 

3 Yards 

Draw and fire 4 rounds 

8 seconds 

2 rounds Strong Hand, 2 rounds weak Hand supported 

7 Yards 

Draw and fire 2 rounds 

4 seconds 

Strong Hand Supported 

7 Yards 

Draw and fire 2 rounds 

4 seconds 

Strong Hand Supported 

7 Yards 

Draw and fire 2 rounds, reload and fire 2 rounds 

10 seconds 

Strong Hand Supported 

7 Yards 

Draw and fire 4 rounds 

8 seconds 

2 rounds Strong Hand, 2 rounds weak Hand supported 

15 Yards 

Draw and fire 2 rounds 

4 seconds 

Strong Hand Supported 

15 Yards 

Draw and fire 2 rounds 

4 seconds 

Strong Hand Supported 

15 Yards 

Draw and fire 4 rounds 

8 seconds 

Strong Hand Supported 

15 Yards 

Draw and fire 4 rounds, reload and fire 4 rounds 

20 seconds 

Strong Hand Supported 

15 Yards 

Draw and fire 8 rounds 

20 seconds 

Strong Hand Supported 


Table 7. Navy Handgun Qualification COF 


The qualification scenarios provided the participant with two magazines 
(six rounds in each magazine) for the three and seven yard line COF. The 15-yard line 
had two magazines with 12 rounds in each magazine. The recorded navy handgun 
qualification COF commands were used to administer the qualification. The scene 
selected for this scenario was a picture the author took of a flight deck looking out at sea. 
The image was imported into ISMT and the intention was to help the participant develop 
a navy centric experience of a handgun qualification out at sea. Refer to Appendix E for 
screen captures of every setting in the qualification scenarios. 
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c. 


Unlimited Practice Scenario 


The unlimited practice scenario was created so that all participants could 
practice with an unlimited amount of time. The participants were presented with a three, 
seven and 15-yard line targets on the screen to take practice shots. The control group 
utilized this scenario to conduct dry fire training and practice taking aim at the targets. 
The pistol was disconnected from the ISMT and the CO 2 tank. This procedure ensured 
that the participants in the control group did not receive any immediate feedback from the 
ISMT. The control group only received immediate feedback from the instructor. The 
instructor paid close attention so that a pistol is connected or disconnected based on the 
training phase or if the participant was in the control group versus the experimental 
group. 

This scenario provided an unlimited supply of magazines to the 
experimental group. ISMT does not account for an unlimited supply of magazine but 
provided the maximum allotted magazines with six rounds in each magazine. The author 
decided that in order to maximize training, the magazines should have six rounds instead 
of 12 rounds because it forced the participants in the experimental group to reload the 
pistol frequently. The scenario provided instant feedback to the participant in the 
experimental group. Experimental group participants were able to view a silhouette of the 
targets in a blue box at the top of the projection screen. If the participants in the 
experimental group had a hard time seeing a target silhouette to track the position of the 
shots fired, they were allowed to look at the instructor’s computer monitor. The monitor 
displayed the same information but with a zoomed in view. 

All the participants were asked to square their shoulders to the target by 
shuffling to the right or left. This adjustment ensured that the targets were centered in 
front of the participant when they practiced firing the pistol. The author did not want to 
risk any negative training with the participants by allowing them to stand in one place 
and shoot at a target offset or not directly in front of the shooter. If the instructor allowed 
the participant to aim at a target off to the side rather than directly in front, the participant 
looked at an improper sight picture and sight alignment. In order to ensure the continuity 

of good sight picture and sight alignment, the participants needed to see the same thing 
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every time. Refer to Appendix F for screen captures of every setting to recreate the 
unlimited practice scenario. 

d. Practice Qualification Scenario 

The practice qualification scenario was almost identical to the 
qualification scenario except there were pauses between the three, seven and 15-yard line. 
This scenario had two more sub serials than the qualification scenario. Refer to the ISMT 
manual and Appendix G for information on sub serials. The scenario provided a view of 
the target's silhouette in a blue box for the experimental group. These targets were located 
at the top of the projection screen the above the actual target that the participants fired at. 
If the participants had a hard time seeing the silhouette to keep track of the position of the 
shots fired, they were allowed to look at the instructor’s computer monitor that displays 
the same information but with a zoomed in view. The control group also used this 
scenario but the pistol was disconnected from the ISMT and the CO 2 tank. The instructor 
paid close attention so that a pistol is connected or disconnected based on the training 
phase or if the participant was in the control group versus the experimental group. The 
same amount of magazines and rounds in the qualification scenario applied to the 
practice qualification. The recorded navy handgun qualification COF commands were 
used to administer the qualification. Refer to Appendix G for screen captures of every 
setting of the practice qualification scenario. 

2. Voice Recording Instructions 

The author decided that it was imperative the participant received their basic 
training, control group stimulus training or experimental group stimulus training, and 
instructions for the standard navy handgun qualification COF from a recording. The 
author planned on having 30 participants and each participant had to qualify three times 
each. He felt that he would not be able to deliver the instructions and training the exact 
same way throughout the entire data collection period. Special care was required in the 
creation of the media files and timing the ISMT scenarios so that the targets appeared and 
disappeared in a timely manner. 
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3. Lab Setup 

The ready line and firing line was established by laying duct tape for the firing 
line and masking tape for the ready line. The instructor also posted a copy of the four 
general safety rules so that participants viewed and referenced them at all times. It was 
important to have a separate laptop connected to speakers so that the instructor played the 
playlists independently of the ISMT. Before the experiment began, special care was taken 
with ISMT preparations by calibrating the hit camera after the system startup. After the 
system was up and running and the system calibration was complete then the M9 Berretta 
went through a weapon registration and weapon calibration. At this point the instructor 
began to collect data. Every participant went through a shooter registration in ISMT to 
help keep track of the participants’ four-digit identification. 

4. Data Recording 

Based on lessons learned in a class project, the author decided it was best to have 
several backup systems for data collections. Pictures were taken at the completion of 
every qualification. The instructor took pictures of the results page, all the shot groups, 
all the MPI to CZP groups and all 48 trace profiles for every shot fired in the 
qualification. An Excel spreadsheet also was used to collect data and conduct statistical 
analysis. The instructor printed out the results data after the pictures were taken and the 
data was transferred to Excel. The print outs were filed in case any electronic files 
became corrupted or lost. 

D. MAIN MEASURES 

The main measures for this experiment were the various marksmanship 
performance data that ISMT provided at the end of each qualification and a post 
experiment survey. This section lists each of the performance measures used. It also 
provides a description of how they were scored and indicates whether higher or lower 
scores were better. 
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1. Scores on the Standard Navy Handgun Qualification COF 

These scores are based on points earned depending on which region of the target 
was hit. The more points a participant received for a qualification, the better. 
Marksmanship improvement with this measure was determined when participants 
received higher scores on training (second) qualification than the baseline (first) 
qualification. The same determination was made when a participant received higher 
scores on the retention (third) qualification than the training (second) qualification. Thus, 
positive difference scores (e.g., training scores—baseline scores) indicate improvement. 
Transtar II targets were used for this experiment and were in accordance with small arms 
training and qualificaiton instruction, enclosure seven, section 1A (Chief of Naval 
Operations, 2009). The target was split up into six regions. Region one was worth zero 
points, region two was worth two points, region three was worth three points, region four 
was worth four points, region five and six were worth five points. Refer to Figure 7 for a 
picture of a transtar II target. The image also identifies the regions of the target. 
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Region 1 Outside of Target Silhouette 



Figure 7. Transtar II Target with Region Labels 

2. Diameter Size of the Shot Group 

Marksmanship performance was measured by the diameter size of the three, seven 
and 15-yard line shot group. Diameter size measures variability in shot consistency. All 
shot group size diameters were measured in centimeters and a participant showed 
improvement when the diameter decreased over time. Therefore, negative difference 
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scores (training scores - baseline scores) indicate improvement from baseline to training. 
Special consideration determined it was best to only use three targets that represented the 
three, seven and 15-yard line. The scenarios could not use a new target for every step 
during the COF because ISMT cannot determine a shot group diameter with only two 
rounds. Therefore, the three and seven yard line targets received 12 rounds each, while 
the 15-yard line target received 24 rounds. 

3. Average Shot Group 

Average diameter size measures variability in shot consistency. All shot group 
size diameters were measured in centimeters and a participant showed improvement 
when the diameter decreased over time. Therefore, negative difference scores (training 
scores - baseline scores) indicate improvement from baseline to training. Thus, as in the 
diameter size of shot groups, negative difference scores in the average shot group indicate 
improvement. 

4. MPI to CZP 

The length of mean point of impact to center zeroing point (MPI to CZP) 
measures the average distance of the shot fired to a fixed point on the target. The fixed 
point on the target was determined by the author when he placed it in the center of region 
six, during the creation of all the targets. MPI to CZP was measured for the three, seven 
and 15-yard line. The length of MPI to CZP also was measured in centimeters and a 
participant showed improvement with accuracy when the length decreased over time. 

5. Demographic Survey 

The demographic survey was administered in order to block participants into the 
control and experimental groups. It contained questions that asked about the participants’ 
characteristics, marksmanship skill level and experience. 

6. Post Experiment Survey 

After completing the experiment, participants were asked four open ended 
questions to determine their thoughts about the experiment, how they felt about 
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simulators and if they were willing to replace real marksmanship practice with simulated 
marksmanship practice. Refer to Appendix I for a copy of the post survey. The data 
cannot determine if an improvement occurred in the participant’s opinion because a 
preliminary open-ended survey was not administered. 
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III. RESULTS 


A. DATA PREPARATION AND PRELIMINARY ANALYSES 

This chapter will report results from hypothesis testing for this thesis. The data 
was recorded using Excel and all statistical analysis was conducted using the data 
analysis feature for Excel. This section describes data preparation and preliminary 
analyses. 

1. Data Preparation 

The data preparation for this experiment consisted of calculating the difference 
scores on all the main measures. Three sets of difference scores were calculated: (1) the 
difference between baseline and post training performance (baseline - post training) to 
test hypothesis one; (2) the difference between post training and post retention scores 
(post training—post retention) were calculated to test hypothesis 3; (3) the difference 
between baseline and post retention scores (baseline - post retention) also were calculated 
to explore whether group differences would be stronger after a longer time frame. Two 
sample /-tests of these difference scores were used to determine if there were group 
differences in improvement. Prior to performing the two sample /-tests, F-tests were used 
to check equal variance assumption for all comparisons. When the F-test indicated that 
the equal variance assumption was met, the two-sample /-test assuming equal variances 
was used; otherwise, the two sample /-test assuming unequal variances was used. The 
two-sample /-test assuming equal variances was used for all main measure except two. 
The three-yard line score and the three-yard line MPI TO CZP did not meet the equal 
variance assumption between the retention (third) qualification and the training (second) 
qualification. For these two cases, the two-sample /-test assuming unequal variance was 
used. The normality assumption also was checked. Non-parametric tests (Wilcoxin Rank 
Sum Test) were used when the normality assumption was violated. The results from all 
non-parametric tests had the same pattern as two sample /-tests. Therefore, only two 
sample /-test results are described. For all tests, an alpha of .05 one tailed was used. 
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2. Preliminary Analyses 

Before conducting hypothesis testing, the author first wanted to check that the 
control and experimental groups performed similarly at baseline. A two sample f-test 
assuming equal variance was conducted on the baseline final qualification scores (0-240 
possible points) between the control group and experimental group. Results indicate that 
there is no significant difference between the control and experimental group in baseline 
performance (see Table 8). Therefore, even though there was a group difference in the 
distribution of self-reported marksmanship experience, this result indicates that on an 
objective measure of performance, the groups were evenly matched in ability. 

The next section will take a closer look at marksmanship performance. 


t-Test: Two-Sample Assuming Equal Variances 


Baseline Scores = Week 1 




Control 

■Sim 

Mean 

209.4 

201.8666667 

Variance 

422.2571429 

981.4095238 

Observations 

15 

15 

Pooled Variance 

701.S333333 


Hypothesized Mean Difference 

0 


df 

28 


t St at: 

0.778754951 


P(T<=t] one-tail 



t Critical one-tail 

1.701130908 


P(T<=tj two-tail 



t Critical two-tail 

2.048407115 




Table 8. Baseline Scores Two Sample Assuming Equal Variance t-Test 


This section will provide the means and standard deviations for the baseline, 
training and the retention phase of the experiment. The purpose for reviewing the 
descriptive statistics during each phase was to gain some preliminary determination of the 
progress on each group’s performance. Therefore, formal statistical testing was not 
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conducted in order to keep type II error low. Refer to Table 9 for the descriptive statistics 
during the baseline phase. 


Descriptive Statistics 


Baseline 

Scores 

Ave Shot 

Group Size 

3 Yard Shot 

Group Size 

3 Yard MPI 

to CZP 

7 Yard Shot 

Group Size 

7 Yard MPI 

to CZP 

15 Yard 
Shot Group 

Size 

15 Yard 

MPI to CZP 

Control Group Mean 

209.40 

9.77 

10.69 

2.90 

19.91 

5.81 

50,61 

9.76 

Control Group SD 

20.55 

4.20 

6.43 

1.15 

6.46 

3.28 

27.77 

5.19 

Experimental Group Mean 

201.87 

12.02 

13.57 

2.81 

27.18 

7.45 

54.32 

S.73 

Experimental Group SD 

31.33 

5.92 

8.35 

1.76 

15.10 

4.01 

32.50 

4.01 


Table 9. Baseline Descriptive Statistics 

All of the data collected are measured in centimeters with the exception of the 
scores. The baseline descriptive statistics suggested that the control group was 
performing slightly better than the experimental group, but note the large standard 
deviations. The control group has a higher score and appears to have a smaller shot 
group. The experimental group appears to be slightly more accurate on the three- and 15- 
yard line based on the MPI to CZP. Refer to Table 10 for the descriptive statistics during 
the training phase. 


Descriptive Statistics 


Training 

Scores 

Ave Shot 

Group Size 

3 Yard Shot 

Group Size 

3 Yard MPI 

to CZP 

7 Yard Shot 

Group Size 

7 Yard MPI 

to CZP 

15 Yard 
Shot Group 

Size 

15 Yard 

MPI to CZP 

Control Group Mean 

227.60 

6.75 

6,74 

1.91 

15.12 

4.13 

35.37 

7.19 

Control Group SD 

7.26 

1.82 

2.25 

1.08 

5.52 

1.73 

12.37 

2.S9 

Experimental Group Mean 

225.53 

7.11 

7.91 

1.57 

15.93 

2.89 

33.23 

5.50 

Experimental Group SD 

19.97 

2.74 

3.29 

1.00 

5.73 

2.86 

16,26 

3.49 


Table 10. Training Descriptive Statistics 

The training descriptive statistics suggest that the control group and the 
simulation group performed at similar levels. The descriptive statistics suggest that as the 
distance from the target increased, the experimental group decreased their shot group 
size. The control group did not appear to be as accurate as the experimental group based 
on the length of the MPI to CZP on the three-, seven- or 15-yard line. The experimental 
group had more variability on the scores, indicating wider range in participant 
performance. Refer to Table 11 for the descriptive statistics during the retention phase. 
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Descriptive Statistics 


Retention 

Scores 

Ave Shot 

Group Size 

3 Yard Shot 

Group Size 

3 Yard MPI 

to CZP 

7 Yard Shot 

Group Size 

7 Yard MPI 

to CZP 

15 Yard 
Shot Group 

Size 

15 Yard 

MPI to CZP 

Control Group Mean 

222.07 

7.21 

7.84 

2.41 

16.49 

5.97 

33.48 

7.63 

Control Group SD 

19.14 

2.86 

3.82 

1,21 

8,38 

3.39 

14.71 

3.90 

Experimental Group Mean 

223.53 

7.41 

8,69 

2.28 

15.66 

4.33 

34.45 

8,18 

Experimental Group SD 

17.63 

2.56 

3.91 

1.71 

5.72 

2.59 

3.52 

4.71 


Table 11. Retention Descriptive Statistics 

The retention descriptive statistics suggest that the control group and the 
simulation group have similar marksmanship performance levels. The measurements for 
the shot groups and MPI to CZP are very close but the control group has a larger standard 
deviation for the 15-yard line shot group size. 


3. Results from Hypothesis Testing 

This section addressed results from each of the three hypotheses. 


a. Hypothesis One 

HO: There will be no change in the participants’ marksmanship performance and 
scores in the Standard Navy Handgun Qualification Course of fire (OPNAVINST 
3591. IF, 2009) from the baseline performance to training phase. 

HA: Participants who receive simulation-based training in ISMT will have a 
greater improvement in marksmanship performance and scores in the standard Navy 
Handgun Qualification Course of fire (OPNAVINST 3591. IF, 2009) from the baseline 
performance to the training phases than participants who received current naval 
marksmanship training. 

There were no significant differences in improvement between the control and the 
experimental group for the average shot group size, three-yard line shot group size, three 
yard line MPI to CZP, 15-yard line shot group size and 15-yard line MPI to CZP. 
Therefore, the null hypothesis was retained for these variables. 

There were significant differences in the seven yard line shot group size (p 
= .007) and seven yard line MPI to CZP (p = .048). Refer to Tables 12 and 13 for the 
results of the seven-yard line shot group size and seven-yard line MPI to CZP. A negative 
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score indicates more accurate shooting. For both variables, the experimental group 
improved more than the control group. 



Table 12. Hypothesis One, Seven Yard Line Shot Group Statistics 


t-Test: Two-Sample Assuming Equal Variances 



7 Yard Line Training-Baseline MPI to CZP 




Control 

Sim 

Mean 

-1.68 

-4.553333333 

Variance 

9.391714286 

7.724095238 

Observations 

15 

15 

Pooled Variance 

8.857904762 


Hypothesized Mean Difference 

0 


df 

28 


t St at 

2.643937225 


P(T<=t] one-tail 

0.006637884 


t Critical one-tail 

1.701130908 


P(T<=t) two-tail 

0.013275767 


t Critical two-tail 

2.048407115 


SIG DIFF between Control and Sim Group with P(T<^t) one-tail 

SIG DIFF between Control and Sim Group with P(T<^t) two-tail 


Table 13. Hypothesis One, Seven Yard Line MPI to CZP Statistics 
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b. Hypothesis Two 

HO: Among those who do not meet the qualification standard in the standard 
Navy Handgun Qualification Course of fire (Chief of Naval Operations, 2009) at 
baseline, there will be an equal number of participants in each group who do not meet the 
qualification standard after training. 

HA: Among those who do not meet the qualification standard in the standard 
Navy Handgun Qualification Course of fire (Chief of Naval Operations, 2009) at 
baseline, a greater number of participants in the experimental group will reach 
qualification standard after simulation-based training than in the control group. 

There were only three people in each group that failed during the baseline 
qualification. Three per group is insufficient to conduct a statistical test of any 
significance. Therefore, only descriptive statistics are provided. Refer to Table 14 for a 
percentage failures during the entire experiment. 


Failure Stats 

Base 

Training 

Final 

Percent of Failures for Control 

0.13 

0.00 

0.07 

Percent of Failures for Sim 

0.13 

0.07 

0.07 


Table 14. Percentage of Failures 


A closer look at scores for all the failed participants using descriptive statistics 
suggests that this hypothesis should be tested in a future study with appropriate sample 
size. Refer to Figure 8 for a graphical depiction of the mean differences in score during 
the training and retention qualification among those participants who failed at baseline. 
Positive values indicate improvement in performance. 
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Figure 8. Hypothesis Two Mean Difference in Scores by Group and Week 


The participants that failed in the experimental group performed higher than the 
control group both after training and after retention. These descriptive statistics suggest 
that the experimental group enhanced their marksmanship skill to a higher degree with 
simulation-based marksmanship training program. It also suggests that they were able to 
retain more knowledge and ability over a one-week period of time with no training. 

To explore if improvement among participant who failed at baseline was focused 
on particular performance measures, descriptive statistics on the individual performance 
measures were examined. These exploratory analyses indicated that the greater 
improvement in the experimental group is seen in the seven-yard line measures. Refer to 
Figure 9 for a bar graph of the seven-yard line improvement from the baseline 
qualification to the training qualification. 
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7 yd Shot Group 



■ 7 yd Shot 
Group 



Figure 9. Hypothesis Two Mean Difference in Seven-yard Line Performance from 

baseline to post training. 


The descriptive statistics did not indicate that the participants that failed in the 
experimental group were able to retain more marksmanship skills then the participants 
that failed in the control group, after a week of no further training. This may indicate that 
a one-week period of time is not long enough to make that determination. Another, 
analysis using descriptive statistics was done comparing the marksmanship performance 
between the baseline qualification and the retention qualification. The seven-yard line 
seems to indicate that the participants that failed in the experimental group improved 
more than the participants that failed in the control group. Refer the Figure 10 for a bar 
graph of the seven-yard line improvement from the baseline qualification to the retention 
qualification. 
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Figure 10. Hypothesis Two Mean Difference in Seven-yard Line Performance. 


The best example of all the participants that failed was in the experimental group. 
This participant failed the baseline qualification with a low score of 118 points. The 
participant failed the baseline and the training qualification and finally passed the 
retention qualification toward the end of the experiment with 192 points. It is important 
to remember that no significant statistical information can be determined with only one 
person, but the level of improvement is noteworthy. Refer to Figure 11 for a graphical 
representation of this participant’s performance over time. The scale for the scores is 
based on points; more points over time indicated better performance. The scale for all 
other marksmanship performance data is in centimeters and smaller measurements over 
time indicate better accuracy. 
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Figure 11. Most Improved Participant 


c. Hypothesis Three 

HO: Participants in each group will be equally likely to maintain their 
marksmanship performance and scores on the standard Navy Handgun Qualification 
Course of fire (Chief of Naval Operations, 2009) one week after the training day. 

HA: Participants in the experimental group will be more likely than those in the 
control group to maintain their marksmanship performance and scores on the standard 
Navy Handgun Qualification Course of fire (Chief of Naval Operations, 2009) one week 
after the training day. 

There was no significant difference between the control group and experimental 
group based on their performance from the training qualification to retention 
qualification. This experiment failed to reject the null hypothesis three. 

4. Exploratory Analyses 

One week with no further training may not be enough time to determine any loses 
of marksmanship performance. According to the qualification criteria for pistols found in 
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Enclosure three, section five, paragraph A of the small arms training and qualificaiton 
instruction (Chief of Naval Operations, 2009) a sailor can go as long as 30 day after 
training before a qualification; “The period of time between the 
marksmanship/safety/weapons familiarization training and live fire qualification should 
not exceed 30 days.” (p. 2) 

To explore whether group differences in improvement occurred between baseline 
and the retention phase, two sample /-tests assuming equal variances were conducted on 
the difference scores for each performance measure. Significant group differences in 
improvement on the seven-yard line shot group (p = .02) and seven-yard line MPI to CZP 
(p = .003) were significant. The lower the diameter in the shot group, the more consistent 
the participant became over time. The lower the length of MPI to CZP the more accurate 
the participant became over time (refer to Table 15). Thus, for both variables, negative 
scores indicate improvement. In both cases, the experimental group improved to a greater 
extent than the control group. 


t-Test: Two-Sample Assuming Equal Variances 

r 



t-Test: Two-Sample Assuming Equal Variances 



7 Yard Line Retention - Baseline Group Size 




7 Yard Line Retention - Baseline MPI to CZP 




Control 

Sim 



Control 

Sim 

Mean 

-3.42667 

-11.52 


Mean 

0.16 

-3.11333 

Variance 

93.54495 

125.6746 


Variance 

10.01829 

7.511238 

Observations 

15 

15 


Observations 

15 

15 

Pooled Variance 

109.6098 



Pooled Variance 

8.764762 


Hypothesized Mean Difference 

0 



Hypothesized Mean Difference 

0 


df 

28 



df 

28 


tStat 

2.117061 



tStat 

3.027965 


P(T<=t) one-tail 

0.021635 



P(T<=t) one-tail 

0.002621 


t Critical one-tail 

1.701131 



t Critical one-tail 

1.701131 


P(T<=t) two-tail 

0.043271 



P(T<=t) two-tail 

0.005241 


t Critical two-tail 

2.048407 



t Critical two-tail 

2.048407 


SIG DIFF between Control and Sim Group with P(T<=t) two-tail 

SIG DIFF between Control and Sim Group with P(T<=t) two-tail 


SIG DIFF between Control and Sim Group with P(T<=t) two-tail 

SIG DIFF between Control and Sim Group with P(T<=t) two-tail 


Table 15. Exploratory Analyses for the Seven Yard Line 

The results of this test suggest that at least two weeks of time is needed to detect 
group differences in improvement. 

B. SUBJECTIVE REPORTS OF THE ISMT EXPERIENCE 

This section reviews input provided by the participants of the experiment, after 
the retention qualification was complete. Every participant was instructed to fill out a 
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post experiment open-ended survey. This survey was an opportunity for the author to get 
a good understanding of how the participants felt about the study and marksmanship 
simulators. Not every participant submitted a post survey and not every participant that 
submitted a post survey had much value, if any, toward comments on their experience in 
the experiment. The quotes that are posted in this section are their own thoughts. In an 
attempt to protect the identity of the participants in the experiment no names shall be 
used in the section. Refer to Appendix I for a copy of the post survey used in this 
experiment. 

1. Comments on the Value of Simulation 

• Without any reservation, I fully believe in the effectiveness of the ISMT in 
helping train and dramatically improve marksmanship performance... I 
think the SMT’s/EST-2000’s are fantastic, especially their ability to 
simulate training scenarios for MP’s, gate guards, etc. I’ve never seen any 
study proving beyond a doubt that these systems improve marksmanship 
“X percent,” but I have no doubt they’re worth the investment. 

• In my personal experience, I have had several occasions in which 
watchstanders had to stand extra watches to cover new shipmates who 
could not get qualified on a weapon due to being deployed. 

• I whole heartedly believe that the ISMT is a useful tool for marksmanship 
training. The Sailor can get instant feedback with deficiencies on their 
fundamentals of marksmanship and correct them on the spot. 

• I feel these types of simulators are extremely useful in areas like 
marksmanship training. This is a low cost (over time) effort that can be 
used immediately, unlike live fire ranges that have higher resourcing 
requirements. 

• Extremely useful, if for nothing else for the money that would be saved and 
for the valuable information and feedback that trainees receive from the 
simulator. 

• ISMT is very useful for a variety of marksmanship training. First, it should 
be used to teach novice shooters before handling a weapon and “wasting” 
live ammunition to familiarize themselves with the “feel” of recoil and 
mechanics of a pistol. Second, sustainment training should utilize the 
ISMT. Even if someone is a good shot, skills deteriorate with time. Using 
the ISMT gets those skills back up to standard before moving to a live 
range for qualification. Finally, shooters of all levels can benefit from the 
feedback provided by the ISMT. Expert performance can be evaluated and 
emulated by others seeking to perfect their abilities. 
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• Use a shock belt to keep the user in a scared mindset...for example if the 
weapon is condition 1 and safety off but pointed in an unsafe direction, or 
if the weapon is holstered with the safety off, etc. Also it felt as if the 
pause between instruction and commencement of firing was too long and 
can affect the results. 

• Very useful. My score improved greatly. There is very little marksmanship 
training in the Navy for the typical sailor so it would be a needed addition 
to the force. 

• Extremely useful. They would cut down on costs dramatically, allow 
commands to tailor a program suited for their members’ needs, and allow 
a safer environment to train new shooters and re-qualify experienced 
shooters while maintaining the fundamentals of marksmanship... Shooters 
will need to feel some of the aspects of shooting that cannot be 
simulatedweapons failures, the shock of a weapon actually firing, and 
some other intangible factors of shooting. However, simulated training can 
drastically improve the efficiency of real training and allow trainers to 
focus on the areas that shooters are struggling on. 

• Highly useful. The fleet needs to train on them prior to wasting ammo. 
The usefulness of being able to step back and evaluate your muzzle 
control in invaluable... not to mention I’m sure cost effective... This is a 
great concept that should be used in the fleet. It will allow more bullets to 
go to the “fight” and allow for a more cost effective way to train... and one 
that will be much easier to get beneficial training. 

• The indoor simulator is a must have. It allows the shooter to work on his 
abilities and for the instructor to have a real world critique of the errors 
that the shooter is making. 

• Yes, because it’s cheaper and it allows for real caption of what a shooter is 
doing wrong. 

• I believe that this program is needed for the issuing of a firearm to 
civilians as well as military training. The better laws we have and training 
for users of firearms the less injuries we would see. There are too many 
Novice shooters that are more harm to themselves and the innocent then 
they are to their targets. 

• I think that using the simulated trainer will be very useful for training. It is 
a training resource that can be utilized frequently simply due to the fact 
that it does not require travel to a range. The data that is collected provides 
useful feedback for the user in terms of what errors are being made and at 
what point in the firing process they need to make corrections. 

• Simulated Marksmanship training would be very useful in providing 
instruction to personnel in areas/units that have a lower need for live fire 
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exercises. Simulation can also provide a safer and less stressful training 
environment for inexperienced shooters. 

I believe that the simulators have a place for training and proficiency. For 
requals, they would be good as well. 

Comments of Real Versus Simulated Training 

Simulators can be used for training, but at some point the real object must 
be handled. Maybe not every qual, maybe every other or every third 
requal. However, I believe that with the simulators the ability exists that 
training is conducted more often, so instead of yearly requals, they can be 
conducted every 6 months, on that time frame every fourth requal should 
be live-fire, while the other three can be simulated. 

Training using this simulation could be provided before the real training, 
giving trainees the opportunity to absolutely understand the subject and 
perform better and safer when they are exposed to the real training. 

I do not believe that simulated training will ever replace 100 percent the 
real training and also to my mind simulation should not replace the real 
training. It can only be a supplementary tool especially before the real 
training. To my mind, reality contains such unpredictable factors that no 
simulation can predict and simulate them. 

It is clear to me that the Navy would be afraid to replace real training with 
a sim and that is unfortunate because with technology the way it is now, 
the ISMT is better than real range training as it provides immediate 
feedback based on real data not just opinion. 

Although I believe there is a great deal of value added to using the 
simulator. The real training is still that., ‘real’. I believe that an individual 
still needs to experience the feel of a real weapon. Simulated training 
however could reduce the amount of real training. 

If they can be used consistently, I believe they can be very useful. Seemed 
very realistic, minus wind, the smell of the powder, and the bum of the 
casings landing on my arms. 

No, because you can never fully simulate the real thing. A shooter will 
always need to practice their shooting ability on the range with live 
rounds. This allows for a true test of the changes made to improve the 
shooters ability using the simulator. 

If the exact conditions of “real” training were able to be duplicated (recoil, 
sound level, etc.) then I would say that, yes, simulated training might be 
used to replace the range training. Never having fired guns under the 
“real” conditions, I have nothing to compare the simulated training to. I 
feel confident however that the simulated training would be a great tool to 
augment training methods already in place. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 


The question this thesis tried to determine is if the ISMT can be used for the 
purpose of improving participants’ current marksmanship performance and score with the 
goal of meeting a qualification standard. A second question addressed was can ISMT be 
used to sustain an acceptable level of performance. 

The concept is simple; until a person has received enough practice discharging 
rounds down range to improve accuracy and proficiency, it is not certain that they will be 
effective with firearms during an actual force protection crisis. To find the time to take an 
entire command to a firing range is difficult with the current ship schedule, unless it is for 
qualification purposes. If the command had a simulator onboard that could allow 
personnel to practice marksmanship skills on a regular basis, while in port or underway, 
then it is reasonable to believe that personnel can and will protect the ship during an 
attack. 

Effective use of a firearm is the most important factor if a threat was detected. 
Personnel are trained to make the proper announcements to alert the crew and give the 
proper commands to an attacker, but if the watch standers cannot hit the target then they 
have failed. A worse scenario is if they miss the target and hit a civilian or friendly 
personnel. By providing the means to practice a perishable skill like marksmanship, the 
Navy may have less incidents or casualties. In this chapter, the author describes logical 
conclusions based on the results of the data collected during the experiment. 
Shortcomings of the experiment are identified, and recommendations on how to improve 
the study and marksmanship training in general are made. Finally, future works stemming 
from the results are suggested. 

A. DISCUSSION 

The results indicated that simulation training improves certain aspects of 
marksmanship performance, specifically, the seven-yard line. Hypothesis one addressed 
whether there was a difference in marksmanship performance when current naval 
marksmanship training was administered to the control group and simulation-based 
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marksmanship training was administered to the experimental group. Evidence for greater 
improvement in the experimental group than the control group was found for the seven- 
yard line performance measures. The experimental group continually showed greater 
improvement on the seven-yard line shot group size and the MPI to CZP length. The 
seven-yard line shot group size and seven-yard line MPI to CZP was able to reject the 
null hypothesis because the size of the target was much smaller than the three-yard line 
target. The seven-yard line was a more difficult task and recorded performance scores 
support the assumption that there was no ceiling effect. 

Given these results, it was somewhat surprising that significant group results were 
not found for the 15-yard line. The 15-yard line shot group size and 15-yard line MPI to 
CZP should have been able to test the null hypothesis. In this experiment it was possible 
that the participants were able to get familiar with smaller targets from the seven-yard 
line target exposure. The participant may have been able to adjust their sight alignment 
and sight picture well enough from the three yard line target to the seven yard line target, 
that it did not make much of a difference when transition from a -seven-yard line target 
to a 15-yard line target. It was also possible that the study should be administered to a 
sample size of 30-60 participants to reduce the likelihood of type I error from the 15-yard 
line hypothesis testing. 

Hypothesis two addressed whether there was a difference in marksmanship 
performance among those who do not meet the qualification standard when current naval 
marksmanship training was administered to the control group and simulation-based 
marksmanship training was administered to the experimental group. Although there was 
not adequate power to test hypothesis two due to insufficient sample sizes, the descriptive 
statistics suggest that simulation training may be more beneficial for poor shooters. The 
margin for improvement once a shooter was established as an expert was very small. 
Experts still showed improvement during the course of the experiment, but the novice 
shooters showed a substantial increase in marksmanship performance. 

Hypothesis three tested addressed whether there was a difference in 

marksmanship performance after one week passed from the date the current naval 

marksmanship training was administered to the control group and simulation-based 
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marksmanship training was administered to the experimental group. The standards of the 
navy handgun qualification may have a lot to do with the inability to make a statistically 
significant differentiation in improvement between the control group and the 
experimental group. However, it is important to note that on average, participants in both 
groups showed improvement since the beginning to the experiment and displayed a trend 
indicative of higher qualification scores, tighter shot groups and smaller MPI to CZP 
lengths. 

One possible reason for the lack of significant results for the three yard line may 
be due to the simplicity of the distance and the size of the target. A majority of the 
participants, regardless of training received, was able to hit the three yard target and score 
60 of 60 possible points for the baseline, training and retention stage of the qualification 
(i.e., a ceiling effect for the three yard line occurred). If a participant did not gain all 
possible points then they were only a few points off. The shot group size and length of 
MPI to CZP would be better determinants of improvement; however a ceiling effect may 
occur with these performance measures as well. The margin of improvement for all 
measures on the three-yard line may not be enough to effectively test the null hypothesis. 

The next section is recommendations for improving the experiment conducted in 
this thesis and identify possibilities of future work. Although results are promising, there 
were some limitations in this study: small sample size, ceiling effect, amount of time to 
test retention was too small and the marksmanship ship training was not as beneficial to 
experts. Future studies should repeat this experiment with a much larger sample size. The 
study may benefit from a different qualification with difficult tasks and more time should 
be allotted when testing for retention of marksmanship skills. A majority of the trends 
and results of this experiment indicate a study with only beginners and novice should 
provide a level of improvement to properly test all hypotheses. 

Research question one did not have any significant difference in all categories 
except for the seven-yard line shot group size and the seven-yard line MPI to CZP. 
Another study should be administered with a bigger sample size. This thesis only used 30 
participants and divided them up into two groups. A sample size of only 15 participants 
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in each group may not have been enough to make a clear distinction of the main effects 
of the different marksmanship training stimulus programs. 

An analysis of the histograms for all marksmanship performance was conducted 
and using non-parametric results (Wilcoxin Rank Sum Test) in which the distributions 
were not normal gave similar pattern of results. It is reasonable to believe that if each 
group had 30-60 participants versus only 15 that the histograms would have looked more 
normal and possibly proved meaningful results. Future studies need much larger samples 
in order to correctly reject a false null hypothesis. The sample size in this thesis may have 
been too low and result in a high type II error rate, incorrectly retaining null hypothesis 
when it’s actually false. 

Because only three participants failed the baseline qualification, the author was 
unable to address research question two. Another study should be administered to 
participants that failed the baseline qualification in order to determine simulation training 
is more beneficial to poor / novice shooters than experienced shooters. Conducting the 
same study with beginners and failures at baseline will help avoid variability in learning 
curves with people that have not used a firearm in a while but have experience with 
firearms and formal marksmanship training. 

Research question three failed to reject the null hypothesis. The retention 
qualification in this experiment was supposed to determine which group would be able to 
retain their marksmanship skills after a week of no training. Another analysis comparing 
the baseline qualification and the retention qualification may able to test the null 
hypothesis based on the results of the seven yard line shot group size and seven yard line 
MPI to CZP. This may indicate that another study should be administered in which the 
retention phase is longer than one week. 

The navy regulation clearly stated that the time between the training and the live 
fire qualification should not exceed 30 days. Thirty days may be excessive and that a 
study of using navy pistol qualification standards should be conducted comparing a two 
week and one month time period of no marksmanship training. This type of future work 
should be able to determine if the duration of time between training and qualification 
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affect retention skill differentially based on the type of training. The next section will 
address all the software technical difficulties, system limitations, and pistol malfunctions 
experienced during this entire thesis. 

B. TECHNICAL DIFFICULTIES AND RECOMMENDATIONS 

1. Failed to Load Magazines and Provide Trace Profiles 

The experiment met several challenges. For example, ISMT was unable to discard 
a round when the participants failed to shoot required rounds before “cease fire.” This 
action caused compounding technical difficulties: ISMT failed to load magazines with 
the proper amount of rounds. 

ISMT failed to update the number of rounds per magazine if the participants 
failed to fire all required rounds. The three-yard line and seven-yard line required two 
magazines with six rounds. The 15-yard line required two magazines that had 12 rounds 
in each magazine. If a participant fired 11 of the 12 rounds required for the seven-yard 
line, then on the 15-yard line the ISMT would reload with a magazine of six rounds vice 
the required 12 rounds per magazine. The ISMT would not reload with a magazine of 12 
rounds until the participant fired one more round at a target. This is a simple error that 
deals with the way ISMT keeps count of the number of shots fired. 

An example of this type of error is when a participant failed to fire two rounds in 
four seconds when the recorded instruction said, “Cease fire.” The ISMT was still 
waiting for one more round to fulfill the requirement of 12 rounds for the three and seven 
yard line. The system was not programed to count that missed round as a zero and 
continue to calculate the marksmanship performance data for the 11 of 12 rounds that 
were actually fired. ISMT also does not provide the simulation user a button to push or 
have some way to manually instruct the ISMT to continue the qualification and count the 
missing round as a zero for points. Surprisingly, when the participant fired all 12 rounds 
at the three or seven yard line target but at the 15-yard line only hit the target 23 of 24 
times then ISMT would provide the needed marksmanship performance data at the end of 
the qualification. 
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2 . 


Provide Trace Profiles 


The experiment met several challenges first, ISMT was unable to discard a round 
when the participants failed to shot required rounds before ceasefire. This action caused 
an inability to provide trace profiles on several occasions and the rapid-fire sequence was 
suspected to be the cause. The navy handgun qualification had a series of instructions that 
required the participants to shoot rapid fire from the holster. For example, the recorded 
instruction may tell the participant to shoot two rounds in four seconds. It was determined 
that ISMT was unable to track the barrel of the pistol effectively. The M9 Berretta was 
equipped with an infrared beam at the end of the barrel and the ISMT had a hit camera to 
trace where the pistol was pointed towards on the projector screen. Once the recorded 
instruction says fire, the participants quickly drew the weapon and fire two rounds. After 
the qualification was complete, the instructors collected the marksmanship performance 
data and noticed that the trace profile feature did not always work. 

The ISMT had four different trace profile failures. First the trace profile would 
simply display no trace data. The second failure was when the trace profile did not 
display the green one second before shot and the purple .2 seconds before shot, but would 
display the blue one second after shot. The third failure was when the trace profile did not 
display the green one second before shot, but did display the purple .2 seconds before 
shot and the blue one second after shot. The last failure was when the trace profile did 
display the green one second before shot, but not the purple .2 seconds before shot and 
the blue one second after shot. 

The author kept track of every time the trace profile had a malfunction. He 
recorded every observation in Excel and at the end of the experiment had 4,224 
observations. A quick calculation indicated that the trace profile failed to work properly 
17 percent of the time. The ISMT tracking system should be modified so that it can track 
the barrel sensor faster in order to help the instructor to train shooters while using the 
trace profile feature. 
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3. Qualification Scenarios 

The creations of the qualification scenarios took several trials. The first trial had a 
separate target for each course of fire. For example, the first set of instructions on the 
three-yard line had the participant fire two rounds in four seconds. A target would appear 
before the recorded instructions said “Fire.” Then the target disappeared when the 
recoded instructions said “Cease fire.” A new target would appear for the next set of 
instructions. This scenario failed to provide any shot group size data at the end of the 
qualification because ISMT could not calculate a shot group with only two rounds for a 
target. This issue was fixed by creating a scenario that had only three targets representing 
the three, seven and 15-yard line. The targets would appear and disappear at the 
beginning and end of the assigned stage of the qualification. This method enabled 
instructors to record all marksmanship performance data that ISMT can provide. 

4. Loss of Marksmanship Performance Data 

Another issue was that the scenario would not provide marksmanship 
performance data if a participant failed to fire all required rounds for each stage of the 
qualification. The navy handgun COF required the participants to fire 12 rounds at the 
three-yard line target, 12 rounds at the seven yard line target and 24 rounds at the 15-yard 
target. If a participant only fired 11 of the 12 rounds required for the three-yard line 
target, then ISMT output would state invalid for shot group size and MPI to CZP. The 
instructor would observe invalid for performance measures if the same action occurred on 
the seven-yard line. Surprisingly, ISMT would provide marksmanship data if the same 
action occurred on the 15-yard line. 

The author figured out a way to improve the scenario and take the limitations of 
ISMT under consideration. A future study should create a scenario exactly like the 
scenario created in the experiment except with a backup target on the side that is smaller 
and different looking from the main target. When the participant failed to fire required 
rounds within the standard time limit, then the instructor should instruct the participant to 
fire a round at the smaller and different looking target located off to the side. The 
instructor should keep track of which round was fired so that upon completion of the 


61 



qualification the instructor can drop that particular shot. ISMT would recalculate all the 
measurements for the shot groups and MPI to CZP data. However, ISMT will not 
recalculate the score and special measures should be taken to prevent participants from 
firing at the target after the command to cease fire. 

5. Inability to Calculate Qualification Scores 

It also was vital that the participant fired the round away from the actual target 
used for the qualification because ISMT would not update the score when the instructor 
dropped the shot upon completion of the qualification. For example, a participant only 
fired one round versus two in four seconds and the author told the participant to fire the 
round at the main target instead of the backup target. Then at the end of the qualification 
when the author drops that particular shot, ISMT will not subtract the points earned. This 
limitation will falsify the actual score that the participant should have earned in real life 
because the simulation was not smart enough to recalculate. It is important to have the 
actual and the backup targets look very different and located as far apart from each other 
as possible to ensure that the participants do not get confused with the instructions. 

The author has not tried a second possible solution. Instead of having two separate 
targets, if the participant were instructed to fire a round pointed away from the projector 
screen, then ISMT may be able to continue with the qualification. The instructor should 
be able to drop the shot at the end of the qualification and have no problems with not 
being able to recalculate the score. Future experiments should try to implement these 
solutions in a pilot study before conducting their actual experiment. 

To resolve this limitation in the experiment for this thesis, the author had to rely 
on previous marksmanship qualification experiences and consider this error as a 
technicality. The author did not derive these solutions until after the experiment had 
already began and did not have time to restart another experiment. The instructor 
considered this particular technicality as an alibi, which meant the participant was 
allowed to fire the round in a reasonable allotted time. 
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6. Error Messages 

The author lost performance data from error messages while trying to print the 
average shot group results page. The author also lost performance data from error 
messages of which the cause was unknown. The error messages during the experiment 
only came at the end of a qualification and during data collection. The author was not 
sure what caused the first error message but took a picture for documentation purposes. It 
is important to note that when the instructor pressed the close button that all data was lost 
and irretrievable. Refer to Figure 12 to view the error message. 


^=^==^= ^ 

MMI Lane Resource Applications 

r ......— ■■ ■ . 1 ■ ■■■ ■ 

MMI Lane Resource Applications has encountered a 
pioblem and needs to close. We are sorry for the 
inconvenience. 

.■■■■■—.— —.. . .- —-- 

If you were in the middle of something., the information you were working on 
might be lost. 

For more information about this error, click here. 

Close 

_ 


Train with the best. 


Figure 12. MMI Lane Resource Applications Error 

ISMT also had printing issues that caused an error message, which resulted in loss 
of data. At the end of every qualification, the instructors would have to print out the 
marksmanship performance data as backup in case electronic soft copies were corrupted 
or lost. This particular ISMT failed to print the actual size of the average shot group size. 
ISMT at first always printed the average shot group size as 1.0 centimeters. The author 
manually wrote the actual size on the print out and filed it. This limitation continued until 
it was discovered that the ISMT would display an error message when the instructor tried 


LANES 

MMI 
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to print the average shot group results. The error message only gave one button to push 
and ISMT would exit the simulation screen and all data was lost. Refer to Figure 13 to 
view the LANEDX MFC Application error message. 



Figure 13. LANEDX MFC Application Error 


The instructor decided to press click here versus the close button and another 
message came up. The author is not sure what the message means but took a picture for 
documentation purposes. Refer to Figure 14 to view the error signature message. 


-"-- 

LANEDX MFC Application 


mmSSSBi 

E rror signature 



AppName: m3lanesp$c exe AppVer: 1.0. C 
ModVef 10 0 1 Offset 0003833^ 

11 ModName: m3lanespsc.exe 


T o view technical Reformation about the error re 

oort click hrifp 

- 1 

innp. 

Close 


Figure 14. Error Signature Message 


The instructor once again decided to press click here versus the close button and 

another message came up. The author is not sure what the message means but took a 
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picture for documentation purposes. Refer to Figure 15 to view the error signature 
message. 



Figure 15. Error Report Contents 


The author found no solutions for this error but came up with a policy that future 
thesis studies should consider. After completion of any qualification, the instructors 
should first record all data onto an Excel file and be extremely careful not to print 
anything out. The instructors should then take a picture of every data screen which 
includes the following; results page, average shot group size, shot group one (three-yard 
line), shot group two (seven yard line), shot group three (15-yard line), MPI to CZP 
group one (three-yard line), MPI to CZP group two (seven-yard line), MPI to CZP group 
three (15-yard line), and all 48 trace profiles. The author also required that all data 
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screens get printed with the exception of the average shot group page because it was the 
source of the error messages. 


7. Menu Navigation 

The next technical difficulty was caused by the instructor that ran the simulation. 
In order to navigate through the ISMT interface, the user had to either push enter or the 
letter Q in order to back out of a screen. The technical difficulty happens at the end of the 
qualification when the instructors are collecting data. If the user pushes the enter button 
and the simulation has “ Continue ” selected, then the ISMT will move to the next screen 
and discard all the marksmanship performance data. It is critical that all data is recorded 
in Excel, all pictures are taken and all data screens are printed out before the Continue 
option is selected and the enter button is pushed. There is no way to undo, or go back to 
previous screen to enable the instructors to collect data again. The same effect will occur 
if the simulation user pushes the Q button at any time. To avoid any loss of data, every 
instructor must be conscious of this limitation and aware of which option they are about 
to select on the screen. 

8. Malfunction of the M9 Berettas 

The last technical difficulty and the biggest issue for this thesis were the seven out 
of commission (OOC) M9 Berettas. The experiment started out with two M9 Berettas and 
only used one to conduct the experiment. The other was used as a prop to help the 
instructor demonstrate the functionality of the pistol to the participants. Given the number 
of participants and the length of each session, this was a lot of wear and tear on the 
pistols. Training sessions began at 0700 in the morning and the last training session 
ended at 1800 in the evening. 

The first pistol with serial number BER4484182 had an automatic slide-lock 

activating mechanism break. This caused the pistol to not lock the slide to the rear when 

the magazine was empty. The second pistol with serial number BER441064Z had a 

safety-on sensor that was too slow. This caused the pistol to randomly and inadvertently 

fire if the hammer is in the single action position (in the full cocked position) and the 

safety was engaged to ride the hammer home. The ISMT still thinks that the safety level 
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was off because the safety-on sensor was too slow and the hammer reached the forward 
position before ISMT had a chance to catch up causing the pistol will fire a round. The 
pistol with serial number BER441064Z also fired a round when the participant was 
chambering a virtual round by pulling on the slide to the rear and releasing the slide 
forward. 

The third pistol with serial number BER408276Z did not detect that the hammer 
had fallen and would not fire a round. The forth pistol with serial number BER422910Z 
had a slow safety-on sensor and exhibited the same symptoms as the pistol with serial 
number BER441064Z. The fifth pistol with serial number BER426414Z often 
mysteriously lost virtual rounds from the magazine without firing the pistol or without 
ejecting a round from the chamber. The simulation user watched the screen and witnessed 
the rounds in the magazine decrease while the weapon was not being fired. 

The sixth pistol with serial number BER412636Z had an automatic slide-lock 
mechanism that would only partially engage. This would cause the pistol to release the 
slide forward with the slightest tap or rapidly moving the pistol in a forward motion and 
stopping. The pistol was an ineffective training tool because the participants never 
experienced the motion of have to press the slide release lever to close the slide since the 
pistol did that for them. The last pistol with serial number BER426445Z had a slow 
safety sensor-on sensor, which had the same symptoms as the previous pistols with the 
same malfunction and would not fire all rounds in the magazine. 

In conclusion, based on the vast number of technical difficulties, the main 
recommendation for future work is to have several data backup systems in place and have 
as many pistols available as possible. Additional recommendations for future work 
should consist of studies that only focus on novice shooters and have much larger sample 
sizes. A study with a more challenging qualification than the one used in the experiment 
should be considered in order to avoid any ceiling affects. 

C. SUMMARY 

This thesis found significant differences between the current naval marksmanship 
training program and a simulation-based marksmanship training program. The results 
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from the seven-yard line portion of the navy handgun qualification COF indicated that 
participants significantly benefited from the simulation-based training program more so 
than the current naval marksmanship training program. An exploratory analysis suggests 
that a time period of at least two weeks is necessary to detect group differences in 
retention of marksmanship skills. Additionally, an exploratory analysis indicates that the 
simulation-based training program may be most beneficial for poorer marksmen. Finally, 
responses from the post experiment survey indicated that the majority of participants in 
the experiment still had reservation about replacing real live fire training with simulation- 
based training, but established that there are benefits to having a marksmanship trainer 
available. 
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APPENDIX A. OPNAVINST3591.1F STAGE 


The contents of this appendix are screen captures of the OPNAVINST3591.1F 
stage. These pictures are provided so that future thesis students can navigate to the stage 
used in this experiment. Readers may also reference the fats manual. The stage was 
named after the OPNAV INSTRUCTION 3591. IF Small Arms Training and 
Qualification. 
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APPENDIX B. 3-YARD ZERO SCENARIO 


The contents of the appendix are screen captures of the 3-yard zero scenario. 
These pictures are provided so that anyone can recreate the course of fire used in this 
experiment. The scenario was established with respect to Navy Handgun Qualification 
Course found in the OPNAV INSTRUCTION 3591. IF Small Arms Training and 
Qualification. 
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APPENDIX C. 7-YARD ZERO SCENARIO 


The contents of the appendix are screen captures of the 7-yard zero scenario. 
These pictures are provided so that anyone can recreate the course of fire used in this 
experiment. The scenario was established with respect to Navy Handgun Qualification 
Course found in the OPNAV INSTRUCTION 3591. IF Small Arms Training and 
Qualification. 
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APPENDIX D. 15-YARD ZERO SCENARIO 


The contents of the appendix are screen captures of the 15-yard zero scenario. 
These pictures are provided so that anyone can recreate the course of fire used in this 
experiment. The scenario was established with respect to Navy Handgun Qualification 
Course found in the OPNAV INSTRUCTION 3591. IF Small Arms Training and 
Qualification. 
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The target travels around the path specified below an infinite number of times. 
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APPENDIX E. UNLIMITED PRACTICE SCENARIO 


The contents of the appendix are screen captures of the unlimited practice 
scenario. These pictures are provided so that anyone can recreate the course of fire used 
in this experiment. The scenario was established with respect to Navy Handgun 
Qualification Course found in the OPNAV INSTRUCTION 3591. IF Small Arms 
Training and Qualification. 
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This tab associates a target's responses (behaviors) to things that occur (events). 


Event 
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The target's exposure r 
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s over Default 

Disappear after the event occurs 1 time (s) using a delay of 0.00 seconds. 
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Select the event 
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[ Add ] 

Select the behavior that is activated by the event above. 


| Update 


0 This behavior is a default behavior for the given event 

This behavior replaces the default behavior for the path node below. 
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This tab configures target exposure timing. 

®;The target appears: 

10.0 seconds after the subserial begins. 

O The target appears after a random amount of time. 

Minimum time Maximum time 

10.0 ~| seconds after the subserial begins |0.0 ~| seconds after the subserial beg 

O The target disappears 
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O The target disappears after a random amount of time. 
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® The target disappears when the subserial ends. 
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This tab associates a target's responses (behaviors) to things that occur (events). 
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0 This behavior is a default behavior for the given event 
O This behavior replaces the default behavior for the path node below 


| Remove 




Errors and Warnings 
Ready 


15-Yard Target Behaviors 



15-Yard Target Scoring 
133 





































































15-Yard Target Regions 


134 
























APPENDIX F. PRACTICE QUALIFICATION SCENARIO 


The contents of the appendix are screen captures of the practice qualification 
scenario. These pictures are provided so that anyone can recreate the course of fire used 
in this experiment. The scenario was established with respect to Navy Handgun 
Qualification Course found in the OPNAV INSTRUCTION 3591. IF Small Arms 
Training and Qualification. 
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APPENDIX G. DEMOGRAPHIC SURVEY 


A Comparison of Current Naval Marksmanship Training VS 
Simulation Based Marksmanship Training with the use of Indoor 
Simulation Marksmanship Trainer (ISMT) 

Purpose: This research is intended to provide naval personnel with the option to using 
simulation for the purpose of basic training necessary to meet a qualification standard. The 
purpose for doing this research is to provide a more robust method for conducting 
marksmanship training in the Navy. 

#_Note: Please choose a four digit number for data recording purposes. Do not use 

the last four of your social. This information will keep your identity anonymous 

Demographic questions: 

Directions: Answer the following questions or circle the answer that applies to you. 

1. Are you (circle one age group): 18-25 26-30 

31-35 36-40 41+ 

2. What is your height?(ex. 5' 8")_ 

3. What is your weight? (180 lbs)_ 

4. Are you a (circle one): Male Female 

5. Are you (circle one): Right Handed Left Handed Both/Ambidextrous 

6. Do you wear corrective lenses, ex. glasses or contacts (circle one): Yes No 

7. Are you color deficient? (circle one): Yes No 

8. Are you (circle one): Active duty Reserves Veteran Civilian 

a. If you are Active duty, which branch of service (circle one): 

Navy Marine Corp Airforce Army International Student 

b. What is your Rank? (ex. LT)_ 

c. Years of Service? (ex. 6yrs)_ 

d. What is your specialty (ex. SWO, Pilot, etc.)? 


9. Have you ever fired a gun? (circle one): Yes No 

a. If you answered yes, describe what kind:_ 

10. How would you rate your level as a marksman right now? (circle one): 

i. Novice 

ii. Marksman 

iii. Sharpshooter 

iv. Expert 

11. Have you ever received any formal marksmanship training? (circle one): Yes No 

a. When was your last training? (MMM/YY)_ 

12. Have you ever fired a gun on a scored course of fire? (circle one): Yes No 

a. If you answered yes, when was the last time: (MMM/YY)_ 

b. What type of gun was used? (ex. 9M qual, M-16 qual)_ 

c. What was your performance level/score? (ex. Expert, or 240)_ 

13. Have you ever used a Marksmanship simulator? (circle one): Yes No 
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APPENDIX H. POST SURVEY 


A Comparison of Current Naval Marksmanship Training VS 
Simulation Based Marksmanship Training with the use of Indoor 
Simulation Marksmanship Trainer (ISMT) 

Purpose: This research is intended to provide naval personnel with the option to using 
simulation for the purpose of basic training necessary to meet a qualification standard. The 
purpose for doing this research is to provide a more robust method for conducting 
marksmanship training in the Navy. 

#_Note: Please choose a four digit number for data recording purposes. Do not use 

the last four of your social. This information will keep your identity anonymous 

Post Critique 

Directions: Write a short answer for the following questions. Please print clearly. 

1. How useful do you think simulators, like the Indoor Simulated Marksmanship Trainer, would 
be for marksmanship training? 


2. Do you think simulated training will ever replace "rear' training? 


3. Explain your experience with using ANY simulations for formal training? (ex. Ship handling, 
simulator, flight simulator, motorcycle simulator) 


4. Any additional comments. 
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